Database Benchmark

Synonym: Database Contest.

See also: "Performance-Vergleich von PostgreSQL, SQLite, db4o und MongoDB" Seminar Database Systems Autumn 2011/2012, Master of Science in Engineering, HSR.

Spatial Database Benchmarks

• HSR Texas Geo Database Benchmark
  • Comparing PostGIS, SpatiaLite and Geocouch
  • Comparing PostGIS with Lucene/Solr Seminar Database Systems Autumn 2013/2014, Master of Science in Engineering, HSR.
• Spatial Overlay/Clipping:
  • "ArcGIS vs QGIS etc Clipping Contest Rematch revisited": [1]
  • Clipping Contest with Spatialite: [2]

About Database Performance Benchmarking

Existing DB-Benchmarks:

• TPC-C for OLTP benchmarks [3].
  • TPC-R & TPC-H (formerly TPC-DS) for data warehouse & decision support systems.
  • TPC-W benchmark for Web-based systems.
• "The Engineering Database Benchmark" [4].
• "Open Source Database Benchmark" [5].
• PolePosition open source database benchmark [6].

Guidelines

A database performance benchmark hast to consider following different aspects:

1. Cold and warm start (beware that in case of warm start chaching will take place!).
2. Equality and Range Queries.
3. Query-Result Sets, which respond with one tupel or which respond more than half of the tupels in the dataset.
4. Single User versus Multi-user.

Software (Scritps) for benchmark automation:

• PostgreSQL hstore Benchmark - Benchmarking in Python by Michel Ott, 2011.
• pgbench - Benchmark tool for PostgreSQL.

Weblinks / References

• "The Engineering Database Benchmark", Rick Cattell, PDF.
• Open Source Database Benchmark
• http://www.delicious.com/sfkeller/database+benchmark and http://www.delicious.com/tag/database,benchmark/alltime