PEMAVOR Search Trends Tool: How is it done?

We used Google Trends Data in the past for a lot of projects. One big limit of the Google Trends Service is that you can only request up to 5 different keywords. This was the main reason for searching for a scalable solution and some other nice to haves.

All this ended now in a Google Trends Chart Creator – that produces embeddable HTML Code you can use easily to add pretty looking search trends visualizations to your website. Of course we had some internally discussions about the definition of “pretty”. In my (first) opinion this was the best way how to show the search trend data. But I had to agree that YTD and Previous Period perspectives might not be familiar right away to everybody. Finally we decided that you get the most fun with Google Trends Data when you combine it with some nice eye candy 🙂

No official Google Trends API - PyTrends to the rescue!

On big keyword lists it is no option to fetch the google trends data manually. There is no official API available but luckily there is already a python solution there that is doing exactly what we need. Thanks PyTrends! This module is making the browser requests to the google trends website and is parsing the results. There are different API methods available that return the different data you know from the Web UI. For our application we used the "Interest over Time" method that returns the indexed search volume over time for each keyword.

How to get Google Trends data for more than 5 keywords?

Google is showing indexed search data and no absolute numbers. For that reason it is not possible to just loop over a big keyword list and merge the results in the end. We have to adjust the way we are requesting the google trends service to get comparable search trends data across bigger keyword lists: Always keep one keyword static and request with 4 changing keywords from your big list. By doing this we can calculate a global search index based on our fixed base line keyword.

Python Microservice for the data processing - JavaScript for the data visualization

We realized that playing around with Trends data is a lot of fun and running always a local script is not the way to go. For that reason we put everything related to the data fetching and processing in one Python Microservice published to Google Cloud Functions. So far we faced no problems of getting blocked because of too many requests - so currently there is no need to use proxy servers. The Python Service is called with the keyword list you want Trends data for and parameters like "Country" and "Time Period". The response is a JSON object that can be visualized easily with some JavaScript.

More Similar Posts