Performance Tuning Best Practices

Jay Pipes
Community Relations Manager, North America
(jay@mysql.com)

TELECONFERENCE: please dial a number to hear the audio portion of this presentation.

Toll-free US/Canada: 866-469-3239 Direct US/Canada: 650-429-3300 :

0800-295-240 Austria 0800-71083 Belgium
80-884912 Denmark 0-800-1-12585 Finland
0800-90-5571 France 0800-101-6943 Germany
00800-12-6759 Greece 1-800-882019 Ireland
800-780-632 Italy 0-800-9214652 Israel
800-2498 Luxembourg 0800-022-6826 Netherlands
800-15888 Norway 900-97-1417 Spain
020-79-7251 Sweden 0800-561-201 Switzerland
0800-028-8023 UK 1800-093-897 Australia
800-90-3575 Hong Kong 00531-12-1688 Japan

EVENT NUMBER/ACCESS CODE: 921155876

Copyright MySQL AB The World’s Most Popular Open Source Database 1

 MySQL: The World’s Most Popular
Open Source Database

Founded in 1995; operations in 23 countries

Fastest growing relational database
Over 8,000,000 installations; 40,000 downloads / day
Dramatically reduces Total Cost of Ownership (TCO)
Used by leading IT organizations and ISVs

Copyright 2005 MySQL AB The World’s Most Popular Open Source Database 2
 Second Generation Open Source
• MySQL AB is a profitable company
– Develops the software in-house; community helps test it
– Owns source code, copyrights and trademarks
– Targets the “commoditized” market for databases
• “Quid Pro Quo” dual licensing for OEM market
– Open source GPL license for open source projects
– Cost-effective commercial licenses for commercial use
• Annual MySQL Network subscription for Enterprise and Web
– Per server annual subscription
– Includes support, alert and update advisors, Knowledge Base,
Certified/Optimized Binaries
• MySQL supports it users
– Worldwide 24 x 7 support
– Training and certification
– Consulting “Reasoning's inspection study shows that the
code quality of MySQL was six times better
than that of comparable proprietary code. ”
Reasoning Inc.

Copyright MySQL AB The World’s Most Popular Open Source Database 3

 Overview
● Profiling and Benchmarking Concepts
● Sources of Problems
● Indexing Guidelines
● Schema Guidelines
● Coding Guidelines
● Server Parameters

Copyright MySQL AB The World’s Most Popular Open Source Database 4

 Benchmarking Concepts
● Provides a track record of changes
➢ Baseline is the starting point

➢ Testing done iteratively

➢ Deltas between tests show difference that the

change(s) made
● Stress/Load testing of application and/or database
● Harness or framework useful to automate many
benchmark tasks

Copyright MySQL AB The World’s Most Popular Open Source Database 5

 Benchmarking Tips
● Always give yourself a target
● Record everything
✔ Schema dump
✔ my.cnf files
✔ hardware/os configuration files as needed
● Isolate the problem
✔ Shut down unnecessary programs
✔ Stop network traffic to machine
✔ Disable the query cache
✔ Change one thing at a time

Copyright MySQL AB The World’s Most Popular Open Source Database 6

 Benchmarking Toolbox
● SysBench
➢ http://sysbench.sourceforge.net/

● mysqlslap (5.1+)
➢ http://dev.mysql.com/doc/refman/5.1/en/mysqlslap.html

● Apache Bench (ab)

● supersmack
➢ http://www.vegan.net/tony/supersmack/

● MyBench
● http://jeremy.zawodny.com/mysql/mybench/

Copyright MySQL AB The World’s Most Popular Open Source Database 7

 Profiling Concepts
● Diagnose a running system
● Low hanging fruit
➢ Diminishing returns

➢ Be careful not to over-optimize

● Identify performance bottlenecks in

➢ Memory

➢ CPU

➢ I/O (Disk)

➢ Network and OS

Copyright MySQL AB The World’s Most Popular Open Source Database 8

 Profiling Toolbox
● SHOW Commands
➢ SHOW PROCESSLIST | STATUS | INNODB STATUS
➢http://dev.mysql.com/show
● EXPLAIN
➢ http://dev.mysql.com/explain

● MyTop
➢ http://jeremy.zawodny.com/mysql/mytop/

● Whole host of Linux power tools

➢ gprof / oprofile

➢ vmstat / ps / top / mpstat / procinfo

● apd for PHP developers

➢ http://pecl.php.net/package/apd

Copyright MySQL AB The World’s Most Popular Open Source Database 9

 Slow Query Log
● Slow Query Log
● log_slow_queries=/var/lib/mysql/slow­queries.log
● long_query_time=2
● Use mysqldumpslow
● (5.1+) Can log directly to a table, plus does not require
restart of server
● SET GLOBAL SLOW_QUERY_LOG = { ON | OFF }
● http://dev.mysql.com/doc/refman/5.1/en/log-
tables.html

Copyright MySQL AB The World’s Most Popular Open Source Database 10

 Profiling Tips
● Get very familiar with EXPLAIN
➢ Access types

➢ Learn the type, key, ref, rows, Extra columns

● Low hanging fruit (diminishing returns)

● Use MyTop to catch locking and long-running queries
in real-time

Copyright MySQL AB The World’s Most Popular Open Source Database 11

 Sources of Problems
● Poor or nonexistent indexing
● Inefficient or bloated schema design
● Bad SQL Coding Practices
● Server variables not tuned properly
● Hardware and/or network bottlenecks

Copyright MySQL AB The World’s Most Popular Open Source Database 12

 Indexing Guidelines
● Poor or missing index fastest way to kill a system
● Ensure good selectivity on field
● Look for covering index opportunities
● On multi-column indexes, pay attention to the order of
the fields in the index (example ahead)
● As database grows, examine distribution of values
within indexed field
● Remove redundant indexes for faster write
performance

Copyright MySQL AB The World’s Most Popular Open Source Database 13

 Common Index Problem

// This top query uses the index
CREATE TABLE Tags (
// on Products2Tags
  tag_id INT NOT NULL AUTO_INCREMENT
, tag_text VARCHAR(50) NOT NULL
SELECT p.name
, PRIMARY KEY (tag_id)
, COUNT(*) as tags
) ENGINE=MyISAM;
FROM Products2Tags p2t
INNER JOIN Products p
CREATE TABLE Products (
ON p2t.product_id = p.product_id
  product_id INT NOT NULL AUTO_INCREMENT
GROUP BY p.name;
, name VARCHAR(100) NOT NULL
// many more fields...
// This one does not because
, PRIMARY KEY (product_id)
// index order prohibits it
) ENGINE=MyISAM; 
SELECT t.tag_text
CREATE TABLE Products2Tags (
, COUNT(*) as products
  product_id INT NOT NULL
FROM Products2Tags p2t
, tag_id INT NOT NULL
INNER JOIN Tags t
, PRIMARY KEY (product_id, tag_id)
ON p2t.tag_id = t.tag_id
) ENGINE=MyISAM;
GROUP BY t.tag_text;

Copyright MySQL AB The World’s Most Popular Open Source Database 14

 Common Index Problem Solved

CREATE TABLE Tags (
  tag_id INT NOT NULL AUTO_INCREMENT
, tag_text VARCHAR(50) NOT NULL
, PRIMARY KEY (tag_id)
) ENGINE=MyISAM;
CREATE INDEX ix_tag 
ON Products2Tags (tag_id);
CREATE TABLE Products (
  product_id INT NOT NULL AUTO_INCREMENT
// or... create a covering index:
, name VARCHAR(100) NOT NULL
// many more fields...
CREATE INDEX ix_tag_prod
, PRIMARY KEY (product_id)
ON Products2Tags (tag_id, product_id);
) ENGINE=MyISAM; 
// But, only if not InnoDB... why?
CREATE TABLE Products2Tags (
  product_id INT NOT NULL
, tag_id INT NOT NULL
, PRIMARY KEY (product_id, tag_id)
) ENGINE=MyISAM;

Copyright MySQL AB The World’s Most Popular Open Source Database 15

 Schema Guidelines
● Inefficient schema another great way to kill
performance
● Use the smallest data types necessary
➢ Do you really need that BIGINT?

● Normalize first, denormalize only in extreme cases

Copyright MySQL AB The World’s Most Popular Open Source Database 16

 Schema Tips
● Consider horizontally splitting many-columned tables
(example ahead)
● Consider vertically partitioning many-rowed tables
➢ Merge tables (MyISAM only)

➢ Homegrown

➢ Partitioning (5.1+)

● Fewer fields = Narrow rows = More records per block

● Use “counter” tables to mitigate query cache issues
(example ahead)
➢ Essential for InnoDB

Copyright MySQL AB The World’s Most Popular Open Source Database 17

 Horizontal Partitioning Example
CREATE TABLE Users (
  user_id INT NOT NULL AUTO_INCREMENT
, email VARCHAR(80) NOT NULL
CREATE TABLE Users (
, display_name VARCHAR(50) NOT NULL
  user_id INT NOT NULL AUTO_INCREMENT
, password CHAR(41) NOT NULL
, email VARCHAR(80) NOT NULL
, PRIMARY KEY (user_id)
, display_name VARCHAR(50) NOT NULL
, UNIQUE INDEX (email)
, password CHAR(41) NOT NULL
) ENGINE=InnoDB;
, first_name VARCHAR(25) NOT NULL
, last_name VARCHAR(25) NOT NULL
CREATE TABLE UserExtra (
, address VARCHAR(80) NOT NULL
  user_id INT NOT NULL
, city VARCHAR(30) NOT NULL
, first_name VARCHAR(25) NOT NULL
, province CHAR(2) NOT NULL
, last_name VARCHAR(25) NOT NULL
, postcode CHAR(7) NOT NULL
, address VARCHAR(80) NOT NULL
, interests TEXT NULL
, city VARCHAR(30) NOT NULL
, bio TEXT NULL
, province CHAR(2) NOT NULL
, signature TEXT NULL
, postcode CHAR(7) NOT NULL
, skills TEXT NULL
, interests TEXT NULL
, company TEXT NULL
, bio TEXT NULL
, PRIMARY KEY (user_id)
, signature TEXT NULL
, UNIQUE INDEX (email)
, skills TEXT NULL
) ENGINE=InnoDB;
, company TEXT NULL
, PRIMARY KEY (user_id)
) ENGINE=InnoDB;

Copyright MySQL AB The World’s Most Popular Open Source Database 18

 Horizontal Partitioning Benefits
● Main table has narrow rows, so...
✔ More records fit into a single data page

✔ Fewer reads from memory/disk to get same number

of records
● Less frequently queried data doesn't take up memory
● More possibilities for indexing and different storage
engines
➢ Allows targeted multiple MyISAM key caches for hot

and cold data

Copyright MySQL AB The World’s Most Popular Open Source Database 19

 Counter Table Example

CREATE TABLE Products (
  product_id INT NOT NULL AUTO_INCREMENT
, name VARCHAR(80) NOT NULL
CREATE TABLE Products (
, unit_cost DECIMAL(7,2) NOT NULL
  product_id INT NOT NULL AUTO_INCREMENT
, description TEXT NULL
, name VARCHAR(80) NOT NULL
, image_path TEXT NULL
, unit_cost DECIMAL(7,2) NOT NULL
, PRIMARY KEY (product_id)
, description TEXT NULL
, INDEX (name(20))
, image_path TEXT NULL
) ENGINE=InnoDB; // Or MyISAM
, num_views INT UNSIGNED NOT NULL
, num_in_stock INT UNSIGNED NOT NULL
CREATE TABLE ProductCounts (
, num_on_order INT UNSIGNED NOT NULL
  product_id INT NOT NULL
, PRIMARY KEY (product_id)
, num_views INT UNSIGNED NOT NULL
, INDEX (name(20))
, num_in_stock INT UNSIGNED NOT NULL
) ENGINE=InnoDB; // Or MyISAM
, num_on_order INT UNSIGNED NOT NULL
, PRIMARY KEY (product_id)
// Getting a simple COUNT of products
) ENGINE=InnoDB;
// easy on MyISAM, terrible on InnoDB
SELECT COUNT(*)
CREATE TABLE ProductCountSummary (
FROM Products;
  total_products INT UNSIGNED NOT NULL
) ENGINE=MEMORY;

Copyright MySQL AB The World’s Most Popular Open Source Database 20

 Counter Table Benefits
● Critical for InnoDB because of complications of MVCC
● Allows query cache to cache specific data set which
will be invalidated only infrequently
● Allows you to target SQL_NO_CACHE for SELECTs against
counter tables, freeing query cache
● Allows MEMORY storage engine for summary
counters, since stats can be rebuilt

Copyright MySQL AB The World’s Most Popular Open Source Database 21

 Schema Tips (cont'd)
● Ensure small clustering key (InnoDB)
● Don't use surrogate keys when a naturally occurring
primary key exists
● Example (of what not to do):

CREATE TABLE Products2Tags (
  record_id INT UNSIGNED NOT NULL AUTO_INCREMENT
, product_id INT UNSIGNED NOT NULL
, tag_id INT UNSIGNED NOT NULL
, PRIMARY KEY (record_id)
, UNIQUE INDEX (product_id, tag_id)
) ENGINE=InnoDB;

Copyright MySQL AB The World’s Most Popular Open Source Database 22

 Coding Guidelines
● Use “chunky” coding habits (KISS)
● Use stored procedures for a performance boost (5.0+)
● Isolate indexed fields on one side of equation
(example ahead)
● Use calculated fields if necessary (example ahead)
● Learn to use joins (!)
➢ Eliminate correlated subqueries using standard joins

(examples ahead)
● Don't try to outthink the optimizer
➢ Sergey, Timour and Igor are really, really smart...

Copyright MySQL AB The World’s Most Popular Open Source Database 23

 Isolating Indexed Fields Example
✔ Task: get the Order ID, date of order, and Customer ID
for all orders in the last 7 days

// Better idea
SELECT *
FROM Orders
WHERE 
// Bad idea order_created >= CURRENT_DATE() – INTERVAL 7 DAY;
SELECT *
FROM Orders // Best idea is to factor out the CURRENT_DATE
WHERE  // non­deterministic function in your application
TO_DAYS(order_created) –  // code and replace the function with a constant.
TO_DAYS(CURRENT_DATE()) >= 7; // Now, query cache can actually cache the query!
SELECT order_id, order_created, customer_id
FROM Orders
WHERE order_created >= '2006­05­24' – INTERVAL 7 DAY;

Copyright MySQL AB The World’s Most Popular Open Source Database 24

 Calculated Fields Example
✔ Task: search for top-level domain in email addresses

// So, we enable fast searching on a reversed field
// value by inserting a calculated field
ALTER TABLE Customers
// Initial schema
ADD COLUMN rv_email VARCHAR(80) NOT NULL;
CREATE TABLE Customers (
  customer_id INT NOT NULL
// Now, we update the existing table values
, email VARCHAR(80) NOT NULL
UPDATE Customers SET rv_email = REVERSE(email);
// more fields
, PRIMARY KEY (customer_id)
// Then, we make a trigger to keep our data in sync
, INDEX (email(40))
DELIMITER ;;
) ENGINE=InnoDB;
CREATE TRIGGER trg_bi_cust 
BEFORE INSERT ON Customers
// Bad idea, can't use index
FOR EACH ROW BEGIN
// on email field
 SET NEW.rv_email = REVERSE(NEW.email);
SELECT *
END ;;
FROM Customers
WHERE email LIKE '%.com';
// same trigger for BEFORE UPDATE...
// Then SELECT on the new field...
WHERE rv_email LIKE CONCAT(REVERSE('.com'), '%');

Copyright MySQL AB The World’s Most Popular Open Source Database 25

 Correlated Subquery Conversion Example
✔ Task: convert a correlated subquery in the SELECT
clause to a standard join

// Good practice
// Bad practice
SELECT p.name
SELECT p.name
, MAX(oi.price) AS max_sold_price
,  (SELECT MAX(price)
FROM Products p
    FROM OrderItems
 INNER JOIN OrderItems oi
    WHERE product_id = p.product_id)
  ON p.product_id = oi.product_id
AS max_sold_price
GROUP BY p.name;
FROM Products p;

Copyright MySQL AB The World’s Most Popular Open Source Database 26

 Derived Table Example
✔ Task: convert a correlated subquery in the WHERE
clause to a standard join on a derived table

// Good performance
SELECT
c.company
// Bad performance
, o.*
SELECT
FROM Customers c
c.company
 INNER JOIN (
, o.*
  SELECT 
FROM Customers c
     customer_id
 INNER JOIN Orders o
   , MAX(order_date) as max_order
  ON c.customer_id = o.customer_id   
  FROM Orders
WHERE order_date = (
  GROUP BY customer_id
 SELECT MAX(order_date)
 ) AS m 
 FROM Orders 
  ON c.customer_id = m.customer_id
 WHERE customer = o.customer
 INNER JOIN Orders o
)
  ON c.customer_id = o.customer_id
GROUP BY c.company;
  AND o.order_date = m.max_order
GROUP BY c.company;

Copyright MySQL AB The World’s Most Popular Open Source Database 27

 Server Parameters
● Be aware of what is global vs per thread
● Make small changes, then test
● Often provide a quick solution, but temporary
● Query Cache is not a panacea
● key_buffer_size != innodb_buffer_size
➢Also, remember mysql system database is MyISAM
● Memory is cheapest, fastest, easiest way to increase
performance

Copyright MySQL AB The World’s Most Popular Open Source Database 28

 Additional Resources
✔ http://www.mysqlperformanceblog.com/
➢ Peter Zaitsev's blog – Excellent material

✔ Optimizing Linux Performance

➢ Philip Ezolt (HP Press)

✔ http://dev.mysql.com/tech-resources/articles/pro-
mysql-ch6.pdf
➢ Pro MySQL (Apress) chapter on profiling (EXPLAIN)

✔ Advanced PHP Programming

➢ George Schlossnagle (Developer's Library)

Copyright MySQL AB The World’s Most Popular Open Source Database 29

 THANK YOU!

● Please email me:

➢ Success stories

➢ War stories

➢ Inventive uses of MySQL

● Feedback on webinar
● Other webinar or article topics you'd like to hear about
● Gripes :)
● Anything else you feel like talking about!
Jay Pipes
(jay@mysql.com)
MySQL, Inc.

Copyright MySQL AB The World’s Most Popular Open Source Database 30