Database Benchmarking for Performance

It’s not hard to estimate how well your databases perform. You know the hardware or service level, the configuration, the resources available to it, and its normal behavior.    

When you need specific performance measures, like total execution time or logical reads per second estimates, it’s time to start benchmarking. It provides you with measurable answers over time. 

Benchmarking in dbWatch Control Center free as part of a free trial or normal performance tuning. 

When is Database Benchmarking Useful? 

Most people use database benchmarking to compare performance objectively. It helps to confirm that a change delivered the expected result, or detect when performance has shifted over time. Here are some examples of when we use it:  

• Before moving a database: Benchmark the existing and proposed environments before a migration. This gives you a direct comparison using the same workload, whether you are moving to the cloud, between cloud providers, or to a different service tier.  

• When evaluating infrastructure: Compare physical and virtual environments, hardware configurations, resource allocations or cloud tiers. Instead of relying only on specifications, you can see which setup actually performs better under the same test.  

• When checking for performance regressions: Run the same benchmark periodically to create a reference point over time. If performance changes significantly, you have measurements showing when and how much it changed.  

• When buying infrastructure as a service: Run the benchmarks regularly, and if the service provider changes the setup, you will know and have evidence. 

How to Benchmark Your Databases in dbWatch  

The database benchmarks in dbWatch Control Center are available to existing users and during the free trial, so you can run them against your own databases and compare the results.  

The dbWatch benchmark runs a defined workload against a database so that you can measure performance under the same conditions each time. The benchmark performs a sequence of database operations — including INSERT, SELECT, UPDATE and DELETE — and measures how quickly the database completes them.   

This repeatable test makes it easier to compare performance, as seen in the case study below. 

An Azure Case Study 

We have been using this approach for quite a while. In 2016, we used dbWatch benchmarks to investigate the performance of Microsoft Azure SQL Database. The resulting report was published in February 2017. Note: This report is historical. The results are not valid today; it’s just a relevant use case.  

The report focused on three service tiers: Basic — 5 DTU, Standard S3 — 100 DT, Premium P2 — 250 DTU; in three areas: West Europe, West Japan, and West US.  We applied the same workload each time we measured. The results show that performance could vary by as much as a factor of five. The results are statistically significant and show that, when performance matters, measuring it matters too. 

You can read the original study below or learn how to set it up.