This article describes how you can use the Custom Metrics feature to capture data from APIs. This could be the API of a technical partner you are using for your business, or your own API that you’ve built to support your own customers or processes. It could even be a custom API you’ve built to collect or calculate your internal business metrics or platform metrics. Test tools and monitoring solutions help you to ensure your APIs work correctly, and this should be a vital part of your platform operations. But you can do much more to gain insight in your platform and business health! Custom API data monitoring doesn’t just check uptime and perform data validation - it captures the data produced by your API, and visually displays how platform and business data progresses over time.
Revisiting API monitoring
You can use test tools like Postman, Insomnia, SoapUI or Swagger UI to test your APIs and inspect data for ad-hoc testing and debugging. You can use a synthetic monitoring tools (like Uptrends’ API monitoring ) to continuously monitor the availability and response times. And not just that - you’ll want to set up in-depth content validation of your JSON data (using JSON expressions) or XML data (using XPath queries) so you can really check that the data returned by the API meets your expectations - without errors, and within acceptable boundaries. This helps to ensure that the processes depending on those APIs continue to work correctly and swiftly. These processes can range from your mobile apps and customer-facing web apps to business processes in your backend.
Tracking API data
Imagine you have a REST API that is part of an e-commerce system that sells products. The entire product catalog consists of several product lines that come from different suppliers, who update their product list several times a day. You have some background processes that update your own product catalog as soon as updates are available (probably using your suppliers’ API, too). Your Product API is directly used by your e-commerce web shop and your mobile app.
Checking that your product list contains the expected number of products and categories (a certain minimum at least) is rather vital for your business. You probably have some reports about that coming from your backend processes, but monitoring it based on your API data gets you the information much faster.
No problem - you can create a Multi-Step API monitor (MSA) that performs one or more calls to your Product API endpoint that returns JSON-formatted data containing the total number of products, the product categories, and the number of products in each category. You add a few assertions that take the individual attributes (let’s look at TotalProductCount and CategoryCount for now) and verify that those numbers don’t exceed their minimum values. For example, TotalProductCount should always be greater than 1500, and CategoryCount should be at least 10. These assertions are safeguards against any unnoticed product update failures that suddenly empty a large part of your product inventory. If something goes wrong, you’ll be alerted by Uptrends and you can act swiftly. That part of your day-to-day Operations is now covered.
Detecting data trends
But there’s more you can do with this data. Each individual monitor check tells you that your product catalog looks healthy (or not). The product and category count numbers were validated during each check, but you can’t clearly see what the actual values were. In fact, it would be nice to see how the number of products fluctuates during the day (micro changes) and during the month and year (macro changes). If the number of products grows faster than you expected, for example, this could point to a problem in removing retired products from your database, which potentially slows down your API, and your mobile app as a result. Detecting this using simple assertions is hard - you need trend analysis!
You’ve already identified your metric TotalProductCount in the MSA assertion, so it’s very easy to now start collecting its data as custom metric data. Let’s set it up!
Set up Custom API monitoring
Once you’ve set up an MSA monitor with a single or multi-step configuration, you must define a
Multi-step monitoring variable
for at least one of your MSA steps. These variables allow you to store specific API response values that can be used anywhere as part of the step definitions. For instance, if you define a variable named UserId, you can use it as part of the step URL as https://your.api.com/GetUserInfo/{{UserID}}.
To create a Customer metric variable:
- Go to your Steps tab.
- Scroll down to go to the Custom metrics section.
- Select the Add metric button.
- In the Metric name, enter a Custom metric name .
- From the dropdown list, pick the option This variable will be filled in script code.
Once done, use the Custom metric variable in the Post-Response script of your step:
-
Select the Post-Response tab of your step.
-
Choose from any of the Custom metrics functions that best fit your needs.
-
Save your monitor to reflect the changes.
-
Add a Custom report tile to easily monitor the Custom metric variable you created.
Within one MSA monitor, you can declare a maximum of five Custom metrics variables. To add more, please get in touch with the Support team for assistance.
Custom metric names
When creating a new custom metric, use a name that is easy to read and provides context to its function. The metrics names and monitor name will appear in the list of available custom metrics once they are added to your custom API data report. Hence, you can use the same custom metric names to represent similar kind of data belonging to different groups (for example, totalSum is used in products versus customer APIs) because you can easily distinguish them from the different monitor names.
Uptrends recommends giving the correct names right away. Renaming a custom metric later will be considered as creating a new and different metric.
Inspecting the data and troubleshooting
Before we start setting up charts and tables, let’s inspect our new data first. Navigate to the Monitor Log, locate your MSA monitor and find a recent check result. Click on it to open up the check result popup.
In the Check details section in the popup, notice how the value(s) for your new custom metric(s) appear below the step results. This gives you direct access to the individual custom metric values as they were captured during the execution of the MSA monitor.
If you don’t see the custom metric value you were expecting to see, consider the following: - Did you accidentally open some check results for an older check that was executed when the new custom metric hadn’t been defined yet? - Is your new custom metric capturing numeric integer data? If it contains text data or non-integer numbers (such as 99.9% or 3.1415) then the custom metric won’t be captured. Only integer numbers are supported right now. Please contact us if you want to discuss your needs. - If something went wrong during the execution of the MSA monitor, the variable that feeds into your custom metric may not have been created at all. Our support engineers can work with you to fix those issues.
Creating reports
When you have verified that your new custom metric is producing proper results, it’s time to create a report that will show the progression of your metric, once you’ve collected enough data. For this example, let’s create a new dashboard ( more on creating dashboards ). Click Dashboards > Add dashboard to create a brand new dashboard. Click on one of the empty tiles, and scroll down in the Add new tile popup to select the tile Custom metrics chart.
The new tile doesn’t display anything at first. Open the tile settings, and locate the monitor that contains the custom metric. Expand it to add the custom metric to your selection. Notice that you can select more than one custom metric: they will be displayed in one chart so you can compare their values.
Similarly, you can use the Custom metrics list tile to create a grid that displays the numeric values for one or more custom metrics. Please note that the displayed values will be average values.
After designing your dashboard, you can save it, export it, and schedule it for automatic delivery via email.
Inspecting raw data
The reports we’ve just looked at will give you a good way to view data trends. But you may want to dig deeper, and work with the underlying data. If you have access to each individual metric value we’ve captured, you can detect sudden spike values more precisely, do min/max/median analysis, and so on.
To do this, create an export file to download all captured metrics to an Excel file. First, navigate to the Monitor Log dashboard to select the appropriate monitor(s). Next, open up the tile settings for the monitor log tile. On the Export tab, make sure to select the Custom metrics checkbox. Click Set to confirm your settings.
Finally, use the dashboard menu (hamburger icon) to start the Excel export process (either directly or through email). Once you have the Excel file, you’ll see separate columns for each custom metric that was captured for your monitor(s).