Mysql query optimization for large database

Question

We have large database, we have nearly two lake records, while we try to search using below query taking too long to bring the result

myquery

SELECT 
      count( DISTINCT e.guid ) AS total
   FROM 
      elgg_entities e
         JOIN elgg_users_entity u 
            ON e.guid = u.guid
         JOIN ( SELECT 
                      subm1.*, 
                      s1.string
                   FROM 
                      elgg_metadata subm1
                         JOIN elgg_metastrings s1 
                            ON subm1.value_id = s1.id ) AS m1 
            ON e.guid = m1.entity_guid
   WHERE 
          m1.name_id = '332'
      AND m1.string LIKE '%96059%'
      AND ( (     e.access_id = -2
              AND e.owner_guid IN ( SELECT guid_one
                                        FROM elgg_entity_relationships
                                        WHERE relationship = 'friend'
                                           AND guid_two =130 )
            )
            OR 
            (     e.access_id IN ( 2, 1, 3, 4, 6, 7 )
              OR ( e.owner_guid = 130 )
              OR (     e.access_id = 0
                   AND e.owner_guid = 130 )
            )
            AND e.enabled = 'yes'
          )

my query explain

Edit, We have more derived queries in the loop so i need more optimization regarding @DRapp

answer

SELECT count( DISTINCT e.guid ) AS total
FROM elgg_entities e
JOIN elgg_users_entity u ON e.guid = u.guid
JOIN (

SELECT subm1 . * , s1.string
FROM elgg_metadata subm1
JOIN elgg_metastrings s1 ON subm1.value_id = s1.id
) AS m1 ON e.guid = m1.entity_guid
JOIN (

SELECT subm2 . * , s2.string
FROM elgg_metadata subm2
JOIN elgg_metastrings s2 ON subm2.value_id = s2.id
) AS m2 ON e.guid = m2.entity_guid
WHERE (
(
subm1.name_id = '332'
AND s1.string LIKE '%10001%'
)
AND (
subm2.name_id = '328'
AND s2.string LIKE '%New York%'
)
)
AND (
(
e.access_id = -2
AND e.owner_guid
IN (

SELECT guid_one
FROM elgg_entity_relationships
WHERE relationship = 'friend'
AND guid_two =2336
)
)
OR (
e.access_id
IN ( 2, 1 )
OR (
e.owner_guid =2336
)
OR (
e.access_id =0
AND e.owner_guid =2336
)
)
AND e.enabled = 'yes'
)
AND (
(
subm1.access_id = -2
AND subm1.owner_guid
IN (

SELECT guid_one
FROM elgg_entity_relationships
WHERE relationship = 'friend'
AND guid_two =2336
)
)
OR (
subm1.access_id
IN ( 2, 1 )
OR (
subm1.owner_guid =2336
)
OR (
subm1.access_id =0
AND subm1.owner_guid =2336
)
)
AND subm1.enabled = 'yes'
)
AND (
(
subm2.access_id = -2
AND subm2.owner_guid
IN (

SELECT guid_one
FROM elgg_entity_relationships
WHERE relationship = 'friend'
AND guid_two =2336
)
)
OR (
subm2.access_id
IN ( 2, 1 )
OR (
subm2.owner_guid =2336
)
OR (
subm2.access_id =0
AND subm2.owner_guid =2336
)
)
AND subm2.enabled = 'yes'
)

Thanks

As best as possible, can you describe what it is you are TRYING to get, and what is "nearly two lake records"? — DRapp
– DRapp, Commented Apr 17, 2012 at 18:45

DRapp · Accepted Answer · 2012-04-12 15:20:36Z

I've restructured your query. Some of the where clauses were redundant (with respect to the e.owner_guid = 130), so superfluous elements removed.

I've added the MySQL clause "STRAIGHT_JOIN" to tell the engine to perform in the order provided by the tables and respective joins. I started with your "m1" as the FIRST prequery to also include your criteria of "name_id" and "String" qualifiers. Ensure your elgg_metadata table has index on the name_id column. Also, since you are not doing anything with any of the other columns from the metadata or meta strings table (except to qualify), I'm only returning the DISTINCT "entity_id". This should return a quick small subset for you.

From that result, only join those pre-qualified to your entities, users and relationships tables (left join on the relationships since that was an "OR" condition later). If it can't find a match on the entity ID, don't bother going any further.

THEN, the rest of the OR criteria can be applied... if the owner_guid = 130 OR found in eer (relationships) via left-join vs an IN (subselect) which would be a performance killer, and your final OR for the Access_ID.

SELECT STRAIGHT_JOIN
      count( DISTINCT e.guid ) AS total
   FROM
      ( SELECT DISTINCT
              subm1.entity_id
           FROM 
              elgg_metadata subm1
                 JOIN elgg_metastrings s1 
                    ON subm1.value_id = s1.id 
           WHERE
                  subm1.name_id = '332'
              AND s1.string LIKE '%96059%' ) AS m1 

         JOIN elgg_entities e
            ON m1.entity_id = e.guid
            AND e.enabled = 'yes'

            JOIN elgg_users_entity u 
               ON e.guid = u.guid

            LEFT JOIN elgg_entity_relationships eer
               ON e.owner_guid = eer.guid_one
               AND eer.relationship = 'friend'
               AND eer.guid_two = 130
               AND e.access_id = -2
   WHERE
         e.owner_guid = 130
      OR eer.guid_one = e.owner_guid
      OR e.access_id IN ( 2, 1, 3, 4, 6, 7 )

Collectives™ on Stack Overflow

Mysql query optimization for large database

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related