SQL : should I insert another column or parse every single row - mysql

Question

Assume millions of lines of traffic data in SQL format.
From the column URL and for each row of given range, I want to get a substring text that matches the target tag.
For example, from the column URL, I have the following texts:
Column: `URL`
Row 1: http://www.google.com/abcdeft?&QQ=123&AA=america&YY=111
Row 2: http://www.google.com/abcdeft?&QQ=123&AA=asia&YY=111
Row 3: http://www.google.com/abcdeft?&QQ=123&AA=africa&YY=111
Row 4: http://www.google.com/abcdeft?&QQ=123&AA=south&YY=111
Row 5: http://www.google.com/abcdeft?&QQ=123&AA=south&YY=111
Row 6: http://www.google.com/abcdeft?&QQ=123&AA=&YY=111
Row 7: http://www.google.com/abcdeft?&QQ=123
...
Row 99999999: http://www.google.com/abcdeft?&QQ=123&AA=ddd&YY=111
Data keep being loaded with lots of updates. So performance does matter. My goal is to:
Identify each row with its unique key-tag &AA=. Basically I need to get the string in the tag &AA= from every single row. For example, I want africa from ~~&AA=africa&~~. None if there is no &AA= but still need to read every single row.
Identify duplicate rows that contain the same tag in &AA=. e.g. row 4 and 5 are duplicates because they have same AA tags of south.
Question: which would be the best way for future data processing?
Option 1. Without URL column
Read every single row in URL column
Parse each row for the tag &AA= using urlparse library
Need a separate script to find duplicate rows with the same AA tag. e.g. using Python, I need to make a list of all items(all tags) and find the duplicate items in the list.
Need a separate query to find the rows that contain duplicate tags. e.g. query the rows that contain the duplicate items in the column URL
Creating separate column specifically for this task seems relatively doable.
Option 2. Insert another new column AA for tag &AA= and start filling out the new column when updating traffic data.
In this way:
No need to Read the column URL
No need to Parse the text in URL to get the tag &AA=
No need to Find duplicate items from one query
- No need to etrieve rows with duplicate items from another query
In this way, we can easily:
Get &AA= data just selecting the column AA
SELECT duplicate rows using COUNT function in SQL
Which one would perform better?

Answer 1

If you can stand the extra space cost of having an additional column then that would be the optimal approach. If there are a lot of duplicates of AA you might consider putting that in another table and then joining to it for queries. That would cut down on the space cost and still give you all the flexiblity. it would make it even easier (faster to query) if you were querying on an ID instead of the textual value of AA.