Optimizing Application Performance with Deep Learning Models

Deep Learning For Application Performance Optimization Dr Zoran Sevarac, Deep Netts @zsevarac, @deepnetts

Vision It would be great to have applications that are able to: learn optimal configuration, self-tune provide best performance in different scenarios

Motivation Performance is always critical Performance tuning must be tailored for Application Load Infrastructure Performance tuning is time consuming, error prone, guesswork When something changes it has to be repeated Ideally, that should be turned into systematic, repeatable, even continuous process

How Performance = F (Load, Resources, Configuration) INPUTS OUTPUTS Load Deep Learning Model Estimated Performance Metrics Resources (CPU, Memory) Configuration Model a dependency between configuration, load, and app performance

Why Modeling a complex, unknown function, with many unknown parameters typical use case for machine learning (regression) Model should predict app performnce for specific load and available resources It can be used to optimize resourceusage or ensure quality of service Task: find configuration settings that will provide performance with optimal usage of resources

Inputs: Load and Configuration Number of concurrent users Request rate (per second) Number of available CPUs Amount of available Memory Additional configuration settings and runtime parameters (numeric and nominal) Anything elsethat s relevant and has impact on application performance

Outputs Performance metrics Average response time (can be skewed with outliers) Apdexscore: (satisfied req+ tolerating requests/2) / totalRequests Min, Maxresponse time (better to discretize) Total execution time (batch processing) Garbage Collection (frequency, time, total time)

General Pipeline 1 2 new loads, usage scanarios Load Test, App Monitoring Data Preprocessing 4 3 tune existing model Evaluation with Test Data Model Building & Tuning 5 6 Deployment to Production Model Monitoring

Data Collection 1. Prepare configurations and loads to test 2. Run loads using different configurations 3. Monitor and record performance metrics 4. Automate/script the process

Apache JMeter Users = Thread Group, Requests = Samplers, Results = Listeners (summary, graphs)

Data Preparation Clean data Filter outliers? Errors , missing values Visualize data Remove correlated inputs Remove inputs that are not correlated with output Prepare data for model building (calculate outputs avg responses (time window size?), Apdex score, etc.) Normalize (scale to range [0,1] or [0.1, 0.9]) Transform nominal inputs to one-hot binary vectors

Feed Forward Neural Network for Regression O U T P U T S I N P U T S A sequence of Layers Each layer performs operations on its inputs and sends outputs to next layer Hidden Activation: TanH or ReLu Output Activation: Linear Loss Function: Mean Squared Error

Network Training Backpropagation Each layer has a number of tunable parameters called weights and training consists of tuning these parameters Training algorithm seeks for a set of layer weights that produce min error a value of Loss Function In every training iteration, algorithm changes all the weights for a small amount that's proportional to the network error - difference between actual and target output.

DEMO: Network Training using Deep Netts

Architecture Tuning How many hidden layers and layer sizes More and bigger layers gives bigger capacity, but possible over-fitting Goal is to find the minimum network size that gives good prediction Automated procedures for tuning: Grid and Random search

Using Trained Network FeedForwardNetwork nnet = FileIO. FileIO.createFromFile( traindedNet.dnet", FeedForwardNetwork .class); float[] inputs = monitoredParams; nnet.setInput(inputs); float[] predictedPerf = nnet.getOutput()

Case Studies Ecommerce Web Application, Tomcat Server 30% Faster Average Response Time with same resources Billing application, batch processing, Web Logic 15% less time to generate 6 million bills, after tuning 8.5h Elastic Search, same performance with 20% less memory

Why Deep Learning For APT Less feature selection and engineering features are learned Good with high dimensions (many input parameters) Good with large amounts of data - more data == better Flexible model (easily supports custom settings)

Deep Learning Tools Neuroph, open source, educational, entry level http://neuroph.sourceforge.net Deep Netts, more advanced options, but easy to use, with specialized wizrads and workflows http://www.deepnetts.com Deep Learning 4J, many advanced options, not so easy to use (requires deeper understanding of deep learning in order to use available options) https://deeplearning4j.org Tensor Flow, Keras, Python based (with Java API), very powerful but not easy to use https://www.tensorflow.org/

Ongoing Development Build a model that will recommend the optimal configuration profile Jenkins pipelines to automate the entire procedure - continiuos performance tuning If conditions are met, trigger the reconfiguration, model rebuilding etc.

Conclusion Machine Learning is already used by many app performance solution providers Deep Learning achieved promising results in our tests It has potential to revolutionize application performance management Got an interesting use case? We d like to hear it! Email: contact@deepnetts.com Twitter: @deepnetts

Oracle Data Science Cloud COMING SOON https://cloud.oracle.com/en_US/ai-platform https://www.datascience.com/ Support for all open source ML libraries Support for Team Collaboration Workflow support for entire lifecycle from data analysis and model building to monitoring Visit them at the booth!

Experts in Modern Development Cloud Microservices and Containers Java, JavaScript/Node.js, PHP, Python DevOps Continuous Delivery Open Source Technologies SQL/NoSQL Databases Machine Learning, AI, Chatbots @groundbreakers developer.oracle.com/ambassador

Learn More http://www.deepnetts.com/intro_to_machine_learning_for_j ava_developers.html http://www.deepnetts.com/deep_learning_for_java_develop ers_in_5_minutes.html https://www.deepnetts.com/product.html#video