The Query: Retrieving, Formatting, And Storing The Data

ALL of the content, custom resources, and general know how, came from the mind and hands of

jnathlich12

I am simply sharing his work with Thwack Nation. (With his approval, of course)

This set of documents will break down the different parts of the resource, and explain (to the best of my abilities) what each piece is, and how each piece works.

How Does This Gizmo Even Work?

Before we can build any charts or graphs, we first need to be able to collect our data. Once the data has been collected, we will need to store that data in a file. However, we cannot simply dump all of the results into a file as they are returned. When saving the data to the file, the data must be properly formatted.

On a high level, in a super basic sense, the custom resource will be doing the following:

1) Query the database to get a result set of data

2) Format the returned data to be read as JSON

3) Save the JSON formatted data to a file

4) Read the saved JSON formatted data as points on the chart/graph

This page will show you how the custom resource is collecting the data you want to graph, as well as what it does to that data once the results are returned.

For the examples on this page, I will be referring to the code in the file attached below. If you would like to start at the finish line, feel free to download the attachment below, and get busy stealin... errr, modifying the code to fit your environment.

The Example

The attached custom resource file, once loaded into your SolarWinds environment, will graph the memory usage of a node. Nothing fancy. Nothing outrageously difficult. No need to be a rocket surgeon or anything. Just a simple example to get us started. We will be collecting a single metric (Average Percent Memory Used), for a single device, over a short time range.

The Query

For the query in this example, we are going to build the date and time fields, and then gather the average memory used for a single node, over the past day.

SELECT
DATEPART(YEAR,DateTime)as Year      --This is where we are building the different parts of the date. These date parts will make up the X-axis
,DATEPART(MONTH,DateTime)as Month
,DATEPART(DAY,DateTime)as Day
,DATEPART(HOUR,DateTime)as Hour
,DATEPART(MINUTE,DateTime)as Minute
,DATEPART(SECOND,DateTime)as Second
,AvgPercentMemoryUsed     --This is the value we are wanting to plot on the chart. This will make up the Y-axis

FROM CPULoad     --This is where we are pulling our data from

WHERE
DateTime >= DATEADD(d, -1, GETDATE())     --This is where we are limiting our results to the past day
AND
NodeID = 123     --This is where we specify the node on which to gather data

ORDER BY DateTime     --This is where we are ordering, or sorting, our data results, which will later be saved to our file in the same order

The Code

Now that we have created, and tested, our query, and verified it is returning the specific data we are wanting to see, we need a way to run that query. While the following code will not actually build any charts/graphs for us, it is enough to create the file that will house the data we want to eventually have plotted on the graph. As this step in this document series focuses on "The Query", we will not be getting into the graphing part until the next step.

For further, more specific, and in depth details on the inner workings of this code, please download jnathlich12's document, HERE. (This can be found in the attachments of the previous/main page, Custom Graphing: Fun With Highcharts)

<%@ Control Language="C#" ClassName="MemoryUseWindows24" Inherits="SolarWinds.Orion.Web.UI.BaseResourceControl" %><%@ Import Namespace="SolarWinds.Orion.Common" %><%@ import namespace="System.Data" %><%@ import namespace="System.Data.SqlClient" %><%@ import namespace="Newtonsoft.Json" %><%@ import namespace="System.IO" %><script runat="server">    protected override string DefaultTitle     //This is where we are giving a name to our new custom resource    {        get { return "Memory Use Last 24 hours"; }     //You can change this name to something else, if you prefer. This is the name that will show up in the resource list when you are choosing a resource to add to the page    }    public void Page_Load()    {  string NetObject = Request.Params["NetObject"];  string NodeID = NetObject.TrimStart('N', ':');

//This is where we are inserting the query we had previously built
        string query = @"SELECT DATEPART(YEAR,DateTime)as Year,  DATEPART(MONTH,DateTime)as Month,  DATEPART(DAY,DateTime)as Day,  DATEPART(HOUR,DateTime)as Hour,  DATEPART(MINUTE,DateTime)as Minute,  DATEPART(SECOND,DateTime)as Second,   AvgPercentMemoryUsed  FROM CPULoad  WHERE DateTime >= DATEADD( d, -1, GETDATE() ) AND DateTime < GETDATE()  AND NodeID = 123  ORDER BY DateTime";        using (System.Data.SqlClient.SqlCommand command = SqlHelper.GetTextCommand(query))        {            grid.DataSource = SqlHelper.ExecuteDataTable(command);  DataTable tblResourceProperties = SqlHelper.ExecuteDataTable(command);  List<int> SeriesList = new List<int>();  foreach (DataRow row in tblResourceProperties.Rows)            {  object measurement = row["AvgPercentMemoryUsed"];  int measurementint = Convert.ToInt32(measurement);  SeriesList.Add(measurementint);            }

//This is where we are formatting our data, to make it valid for plotting on the chart
  string JSONresult;  JSONresult = JsonConvert.SerializeObject(tblResourceProperties);  string[] dataseries;  string output = JsonConvert.SerializeObject(SeriesList);  //lblText.Text = JSONresult;
//This is the location to which we are saving our file. I have simply used the default SolarWinds web folder. You may change this to fit your environment
//You may need to adjust the permissions of the folder to be able to write the file. Otherwise, you can use a different folder, as long as you have write permissions.  string path = @"c:\inetpub\SolarWinds\Orion\OrionImprovement\JSON\Mem24Data.txt";  File.WriteAllText(path, JSONresult);        }  //grid.DataBind();    }</script>

The JSON Formatted Results

Now that we have the code, when the page loads, it should run the query, collect the data, and save the data to a file, formatted properly. Having the data formatted properly is very important. If we do not format the data properly, then our charts will either plot the data points in the wrong spots, or the chart could even fail to load.

Here is a sample of how the data should look , after running the code listed above.

[{"Year":2017,"Month":4,"Day":6,"Hour":9,"Minute":10,"Second":52,"AvgPercentMemoryUsed":19.1795368},{"Year":2017,"Month":4,"Day":6,"Hour":9,"Minute":14,"Second":52,"AvgPercentMemoryUsed":19.1846333},{"Year":2017,"Month":4,"Day":6,"Hour":9,"Minute":18,"Second":52,"AvgPercentMemoryUsed":19.18994},{"Year":2017,"Month":4,"Day":6,"Hour":9,"Minute":22,"Second":52,"AvgPercentMemoryUsed":19.1846333},{"Year":2017,"Month":4,"Day":6,"Hour":9,"Minute":26,"Second":52,"AvgPercentMemoryUsed":19.18462},{"Year":2017,"Month":4,"Day":6,"Hour":9,"Minute":30,"Second":52,"AvgPercentMemoryUsed":19.1847343}]

Okay, But How Do We Make All Of This Work?

Now that we have all of our pieces built properly, and laid out in the correct order, we need to introduce our new custom resource to our SolarWinds environment.

While this step is mentioned in detail on the first/main page, I will simply list the general steps below.

1) Copy your new custom resource file to the appropriate folder on your main SolarWinds web server. (ex. /inetpub/SolarWinds/Orion/NetPerfMon/Resources/Misc/MemoryUseWindows24.ascx)

2) Delete the the resource cache file from your main SolarWinds web server. (ex. C:\ProgramData\Solarwinds\Orion\WebResourceCache.xml)

3) Recycle/Restart the SolarWinds Application Pool on your main SolarWinds web server.

4) Add your new resource to a page, the same way you would add any other resource.

Well, Did It Work?

If all has gone well, and you have made it this far, your work should be now be complete. Or is it? Well, let's do one last step to help us verify all of our hard work has paid off.

So, what are we expecting to see? Well, unfortunately, at this point in the process, we are only expecting to see some boring data in a text file. After you have added the resource to a page, every time that page is loaded, or refreshed, our custom resource should be running through the steps and updating, or rebuilding, our JSON data file. Let's add a simple way to verify the file has the expected data.

To view the contents of the file, simply add the "Custom HTML" resource (the default HTML resource that comes with any standard SolarWinds installation, allowing you to put HTML code directly onto the page), edit that resource, and add the following line of code.

<iframe width="760" height="400" src="/Orion/OrionImprovement/JSON/Mem24Data.txt" frameborder="0" allowfullscreen></iframe>

As long as you have the correct permissions, to view the contents of the folder/file, the contents of the text file should now appear on the page.

Here is a screenshot showing the graph (which we technically have not made yet), as well as the contents of the text file, shown directly under the graph, within the custom HTML resource.

(If you have followed all of the steps to this point, you should actually be seeing a blank custom resource, instead of the graph. We have yet to do any work to create, nor explain, the code to create the graph.)

Uh, What Now?

Well, that is all for this segment. Below, I have attached the completed example, MemoryUseWindows24.ascx, if you would like to jump right into it. Otherwise, if you were following along on your own, you are approximately halfway through creating your custom highcharts resource. The next section will be a direct continuation of this document, and will show you the graphing part of the code. Each of the following sections will specifically relate to the graphing code only, using slight variations to create different types of graphs. From this point on, I will focus more closely on the code for each graph, and less on the details from this page.

While the query will change throughout the following documents, the basics should remain the same.

• Make sure your query works by itself first, testing in SSMS before adding the query to the page.
• Make sure you build a new resource cache after adding any NEW files to the server.
  • (If your file already shows up when you go to add a new resource to the page, any edits you do to that file will appear as soon as you save the file. You do NOT need to rebuild the resource cache file if you are updating existing pages/files)
  • If you plan on using/creating a bunch of custom resources, you might want to add several copies of your resource file into the folder at once, and then just update them as you go. (This will save time by only needing to rebuild cache file one time, instead of each time you add a new file)
    • Change the filename (MemoryUseWindows24.ascx, MemoryUseWindows24-002.ascx, MemoryUseWindows24-003.ascx, etc.)
    • Change the "Default Title" of the resource (CHANGE: get { return "Memory Use Last 24 hours"; }  TO: get { return "Memory Use Last 24 hours 002"; } )
    • Change the location references in each file (Each file needs to save data to its own, unique JSON file name, otherwise each resource will overwrite the others.)