With the move to GA4 imminent, many will be looking to data storage options and specifically linking to BigQuery. We will look at some of the options available and the potential costs.
One of the reasons you need to be thinking about storage options is because there is a 14-month limit on data retention with GA4. So, if you want to be able to access older data, you need to find another way. In addition, the quota limitations imposed with GA4 make it difficult to run reports through Looker Studio, unless you use something like BigQuery. The good news is that you can set up your BigQuery account and get it collecting data right now. Although the paid account still exists, it is available for free with GA4 and offers something which should be more than sufficient for most companies.
Google Cloud Platform account
Before you can link to BQ, you’ll need to set up a Google Cloud Platform account. There are two different account types:
- “Personal” GCP account
This is the quickest and easiest way to get set up. It’s a case of creating your account, adding BigQuery and then linking to GA4.
- Organization / Cloud Identity
The organization setup takes longer but is better in the long run as it future-proofs you against any personnel changes. There are a few steps to setting up with an organization identity:
- Decide Cloud Identity / Google Workspace
- Create a managed user account for your admin user using a generic email address
- Link company domain to Google Cloud (you’ll need IT support for this step)
You can use the Google Cloud set up checklist and then hand over the configuration to the IT support to take things easier. If you just want to get things set up straight away, you can start with a personal account and then transfer things over
You can tell if you have the organization setup if you see your domain on the projects dropdown list.
BigQuery sandbox or paid version?
The next step is to decide between the BigQuery sandbox version or a paid account. The sandbox is a good free option but has certain limitations:
- You cannot have more than 1M daily events
- You don’t get a daily frequency breakdown and can only use streaming
- In addition, tables are set to expire at 60 days. If you then switch to a paid model, you’ll need to remember to set the table expiration date to ‘never’.
- It is very clear when you’re using the sandbox. Each event is one day of data, whereas in the paid version you get an intraday event count.
The paid version requires credit card details for setup and you will need to create a billing account. However, the sandbox works with a personal or organization account so, once you have the resources, you can move your personal account to an organization account.
BigQuery pricing
You can check the Google BQ pricing documentation to see the pricing options. For GA4 purposes, the main uses of BQ will be storage and queries.
For both active and long-term logical storage, you get the first 10GB free each month. Depending on your usage, you will then be charged a small amount per extra GB.
In real terms, what does that look like? Here is an example of real usage from a typical client.
With long-term storage data at only $0.01/GB and active data storage at $0.02/GB, it is worth trying to maintain as much of your data as long-term as possible. “Active” becomes “long-term” 90 days after the data has been put in but becomes active again if it is manipulated. To avoid data passing from long-term back to active and going into a more expensive bracket, you need to avoid doing anything with it besides running queries.
To benchmark potential pricing for different data usage models, you can use this basic pricing calculator. By entering your event data over 7 days, you can work out how many days it would take you to go beyond the free version and how much it is likely to cost per month. As you can see from this example, the cost is very small even if you increase the numbers to cover edge-case scenarios
For queries, the pricing model is similarly reasonable at $5 per TB after the first TB.
As an example, let’s run a query that looks back over 319 days and includes all the data I have collected. Even for a query this size, it is only going to cost 32.50GB.
If you were to run 10 such queries per day, the cost per month would still only be $52. In a normal scenario, and using optimized queries, costs can be kept down considerably.
To look at an example of typical total costs, at Northwoods agency, one of Fred’s reasonably-sized clients has been running BQ for a year and the total annual cost came in at just $32. When looking at this kind of cost, the advantages of using BigQuery should make this an easy decision for most companies.
Takeaways
- You’ll need set up a Google Cloud Platform (GCP) account
- Your account should ideally be an organization resource
- Set up and ink to BigQuery, using either a sandbox or paid account
- You can estimate your costs using Fred’s forecasting calculator
