Google Cloud Platform (GCP) offers a wide array of database services tailored to meet various workload needs, form relational databases like Cloud SQL and Cloud Spanner to No SQL database such as Cloud Firestore and Bigtable.
Each database type suited for specific use cases, making it crucial to understand your data and application requirements before selecting a database.
Relational Databse
Relational databases are ideal for applications requiring structured data with complex relationships, ensuring ACID transactions and string consistency.
Cloud SQL
A fully managed relational database service compatible with SQL server, MySQL and PostagreSQL. It offers automated backups, data replication and disaster recovery.
- Regional Service providing High Availability (99.95%)
- Use SSDs and HDDs (for best performance use SSDs)
- Upto 426 GB of RAM and 30 TB of data storage
Cloud Spanner
A fully managed relational database that provides global consistency and high availability. It is designed for large-scale, global distributed databases.
Use Cases:
- Global E-commerce Platform: Suitable for applications requiring consistent data across multiple regions.
- Financial Services: Ensure high consistency and reliability for financial transations.
Features:
- Cloud Spanner scale horizonatally for reads and writes
- Expensive (compared to Cloud SQL): Pay for nodes & stroage
- Data Export: Use Cloud Console to export data
- Other option is to use Data flow to automate export
- No gcloud export option
NoSQL Databases
NoSQL databases are best suitable for large scale, unstructured or semi-structured data with high scalability and eventual consistency
Cloud Firestore
A fully managed, serverless NoSQL database designed for serverless apps. It supports data access, live synchronization, and built-in security.
Features:
- Highly scalable NoSQL Document Database
- Automatically scale and portitions data as it grows
Use Cases:- Mobile and Web Applications: Ideal for applications requiring offline capabilities and real-time data synchronization.
- Real-Time Analytics: Supports real-time analytics for applications like gaming forums and leaderboards.
- Collaborative Applications: Suitable for multi-user collaborative applications.
Cloud Bigtable
A fully managed NoSQL database service designed for large-scale analytics and data processing. it provides high-throughput reads and writes.
- Use Cases:
- Big Data Analytics: Suitable for big data analytics, IoT data processing, and real-time data processing.
- Financial Analytics: Ideal for financial analytics due to its high performance and scalability.
Features:
- Not serverless: You need to create a server instance (Use SSD or HDD)
- Scale horizonatally with mutiple nodes (No downtime for cluster resizing)
- CANNOT export data using cloud console or gcloud:
- Either use a Java application or use HBase commands
- Use cbt command line tool to work with BigTable (Not gcloud)
- Ex: cbt createtable my-table
Memorystore
- In-memory datastore service: Reduce access times
- Fully managed (Provisioning, Replication, Failover and Patching)
- Highly available with 99.9% availability SLA
- Monitroring can be easily setup using Cloud Monitoring
- Use Redis for low latency access with persistence and high availability
- Gaming Leader Boards, Player Profiles, In memory Stream Processing etc.
BigQuery
Google BigQuery is a fully managed, serverless data warehouse service offered by Google Cloud Platform.
It is designed to handle large-scale data analytics, providing a scalable and efficient way to process and analyze vast amounts of data.
Query external data resources without storing data in bigQuery
When we are talking about a Datawarehouse, importing and exporting data (and formats) becomes very important:
- Load data from a variety of sources, incl. streaming data
- Variety of import formats – CSV/JSON/Avro/Parquet/ORC/Datastore backup
- Export to Cloud Storage (long term storage) & Data Studio (visualization)
- Formats – CSV/JSON (with Gzip compression), Avro (with deflate or snappy compression
Choosing the Right Database
When selecting a database on GCP, consider the following factors:
- Data Structure: Relational databases are best for structured data, while NoSQL databases handle unstructured or semi-structured data.
- Scalability: NoSQL databases generally offer better horizontal scalability.
- Performance Requirements: Consider databases with in-memory caching or high-speed access for demanding workloads.
- Geographic Distribution: Use globally distributed databases for applications serving users worldwide.
- Security and Compliance: Ensure the chosen database meets your security and compliance needs.