Detecting Anomaly in Web Server Logs with Microsoft Azure Cloud – For FREE and at one-tenth the effort!
Every website has web server logs which record the intricate details of site visitors – their browsing behaviors, clicks, actions etc. Web server logs soon become very large and bloated as they log all these information, one record per line. Within this maze of data lies hidden deep secrets about the website. Secrets like what are site visitors actually doing on their website, how best is the server responding to the requests from site visitors, what are the actions that site visitors are taking before they actually convert into buying customers, and many others.
Within the maze of data collected by web server logs, there will undoubtedly be anomalies skewing your most useful information. This is complicated further by the web server logs on your website becoming bloated after recording all of the site visitor’s intricate details, one record per line. This is where Anomaly Detection comes in. Applying this Machine Learning technique allows you to make use of this valuable data, which can include hidden insights into your website, such as what are the site visitors actually doing, how is the server responding to visitor requests, and what actions are visitors taking before they convert into customers.
However, rummaging through the web server logs in search of these hidden insights is not an easy task. The number of such insights that can be gleaned from web server logs increases manifold if Machine Learning techniques like ‘Anomaly Detections’ are applied. Anomaly detection can help to identify fraudulent activities of site visitors, the use of automated robots for malicious purposes and attempted hacking activities among a host of other benefits.
But all these take time, money and dedicated technical resources to make it happen.
Enterprises need to set up the data pipelines, perform the analytics, create the machine learning models, develop the visualizations as well as test and run the whole system. Each of these steps requires skilled professionals who need to work for at least two to three months minimum to make this happen. All these add up to a hefty price tag of anywhere from $50,000 to $100,000 for enterprises to establish on their own web server logs analytics system and perform anomaly detections.
Introducing ThirdEye’s one-click deployable solution for Anomaly Detection in web server logs, generally available on the Microsoft Azure cloud.
ThirdEye Data’s solution works off the web server logs, parses and processes it to get it ready to generate actionable insights.
The solution does the following activities on the web server logs:
- Persist on the Azure Blob Storage,
- Leverage Azure Data Factory for orchestrating the whole end-to-end data processing pipeline.
- Get the data ready at the end of the pipeline, using interactive PowerBI dashboards.
Anomaly Detections modules are inbuilt into the solution, and default machine learning models ready to be used. The data pipeline supports both cold path and hot paths, thereby enabling both batch and real-time analytics. The solution is completely deployable on the Microsoft Azure cloud in less than 15 minutes by clicking on one button. All required components of the solution are automatically created, initiated and populated with custom data as per the end-users requirements. Further customizations take additional time.
ThirdEye is making the base solution available to the open source community while charging for any customizations that may be required. Microsoft Azure charges are billed separately. As and when ThirdEye adds further functionality to the open source solution, they become available to the customers at no additional charges!
Enterprises can now perform Anomaly Detection in their web server logs
– for free and at one-tenth the effort!
