How to Use Google Gemini with Python

AI
LLM API
Google Cloud
The Cloud API, and Model Garden through the Vertex API
Published

May 2, 2025

Google Gemini is a powerful LLM that can be used for various applications.

Learning API is an important skill for any developer and to communicate with software systems will require a lot of flexibility, however Google is a very advanced software company and have built a robust way to communicate with their LLMs which is not quite the same. They even have a system they call discovery which probes all API endpoints in a system and returns a “machine-readable” description of the APIs.

You don’t just want to learn to communicate with a single API designed to communicate with a single LLM model. That model will be replaced in a few months; what you actually want to learn is the industry-standard way to communicate with any LLM via their model garden. That will involve getting acquanted with Vertex AI https://cloud.google.com/vertex-ai/docs?hl=en I will try my best to learn that along with you.

I would ask you to not have it perform tasks that would be easily done without the use of an LLM, if you can write a program that uses 1/100000th of the energy required to ask Gemini, then please do that instead.

What You’ll Need

  • A Google account
  • A Gemini API key (you can get this from the Google Cloud Console)
  • Python 3.7+ installed on your machine

Step 1: Set Up Your Google Cloud Project

  1. Go to the Google Cloud Console.
  2. Create a new project or select an existing one.
  3. Enable the Gemini API for your project.
  4. Create credentials (API key) for your project.
  5. Copy the API key and keep it safe.

Step 2: Install the Required Libraries

There are many ways to communicate with Gemini, but we are going to use an encompassing method through google-cloud-aiplatform. While you could directly communicate through the genai API, learning to do so through the google-cloud library and OAUTH will provide morevisa transferable skills than making http calls.

!pip install google-cloud-aiplatform google-auth google-auth-oauthlib google-auth-httplib2

Step 3: Set Up Authentication

We will use Google OAuth service account to keep a line open with Google. This is just a few more steps from creating the API key and you should be able to navigate it fairly easily knowing that you are making a service account to house your credentials. Download this json and store is securely.

import os
from google.oauth2 import service_account

PROJECT_ID = 'gen-lang-client-0685089672'
LOCATION = 'us-central1'
SERVICE_ACCOUNT_FILE = 'gen-lang-client-0685089672-fe667d84a481.json'  # Normally set this ENV variable outside of the code; here for demonstration
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = SERVICE_ACCOUNT_FILE

credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE)  # vertexai will see this variable
from vertexai.preview.generative_models import GenerativeModel
import vertexai

def init_vertexai(project_id, location="us-central1"):
    vertexai.init(project=project_id, location=location)
    print(f"Initialized Vertex AI with project: {project_id}")

def generate_response(prompt: str, model_name: str) -> str:
    try:
        model = GenerativeModel(model_name)
        response = model.generate_content(prompt)
        return response.text
    except Exception as e:
        print(f"Error generating response: {str(e)}")
        return None
init_vertexai(PROJECT_ID, LOCATION)

model = GenerativeModel("gemini-2.0-flash-001") # Pick a model from the Model Garden

response = model.generate_content("Explain the Google Model Garden, how it may differ from a standard API, and the proper industry-standard techniques around how to implement it securely.")

print(response.text)
Initialized Vertex AI with project: gen-lang-client-0685089672
## Google Model Garden: A Curated Showcase of AI Models

The Google Model Garden is a repository of publicly available AI models, aiming to make advanced AI technology more accessible and usable.  It hosts a diverse range of models, spanning various domains like:

*   **Image recognition and generation:** Models for object detection, image classification, and creating images from text.
*   **Natural Language Processing (NLP):**  Models for text generation, translation, summarization, and sentiment analysis.
*   **Audio processing:**  Models for speech recognition, audio classification, and music generation.
*   **Reinforcement Learning:** Models for training agents to perform tasks in simulated environments.

The models in the Model Garden come from various sources, including:

*   **Google Research:** Cutting-edge research models developed within Google.
*   **Google Cloud AI Platform:** Models trained and deployed on Google Cloud.
*   **Open-source community:** Contributions from researchers and developers outside of Google.

**Key Features of the Model Garden:**

*   **Discovery and Exploration:** Provides a central location to discover and explore pre-trained AI models.
*   **Code Samples and Tutorials:** Offers code examples and tutorials to help users quickly get started with the models.
*   **Integration with Google Cloud:**  Streamlines the process of deploying and using models on Google Cloud Platform (GCP).
*   **Varied Licenses:**  Includes models with different licenses (e.g., Apache 2.0, MIT), catering to various usage scenarios.

## Google Model Garden vs. Standard APIs

While some models within the Model Garden *can* be accessed via APIs, the Model Garden itself isn't simply an API endpoint. Here's how it differs from a standard API:

| Feature        | Google Model Garden                                                                   | Standard API                                                                           |
|----------------|---------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|
| **Purpose**     | Model discovery, exploration, code examples, deployment guidance.                   | Exposing functionality (usually a specific task) through a defined interface.      |
| **Focus**       | Showcasing a *collection* of models, each with potentially different access methods. | Providing a *specific* set of functions with consistent inputs and outputs.         |
| **Access**      | Can involve downloading model weights, deploying custom endpoints, or using pre-built APIs (where available). | Typically accessed through HTTP requests with well-defined input/output formats (e.g., JSON). |
| **Scope**       | Broad, covering many domains and model types.                                       | Narrow, focused on a specific functionality or dataset.                               |
| **Implementation**| Varies greatly depending on the model.                                            |  Usually consistent, following RESTful or other API design principles.                 |

**In essence:** The Model Garden is a catalog, while an API is a specific interface.  The Model Garden *might* point you to an API for a specific model, but its primary purpose is discovery and enabling users to use the models in various ways.

## Secure Implementation of Google Model Garden Models

Securing the implementation of models from the Google Model Garden requires a multi-layered approach.  Because usage can vary (downloading weights, deploying custom endpoints, using APIs), you need to adapt security measures accordingly. Here's a breakdown of best practices:

**1. Secure Model Access and Deployment (if you are deploying the model yourself):**

*   **Principle of Least Privilege:**  Grant the minimum necessary permissions to the service accounts or users that deploy and access the model.  For example, if a model only needs read access to a Cloud Storage bucket, don't grant it broader access.
*   **Managed Identities:** If deploying on GCP (e.g., using Vertex AI), use managed identities for your compute instances (e.g., VMs, containers). This avoids embedding credentials directly in your code or configuration.
*   **Secure Boot:** Use secure boot for your VMs to ensure that only authorized software is loaded during startup.
*   **Confidential Computing:**  For sensitive data, consider using confidential computing solutions to encrypt the model and data in memory during processing. (e.g., Confidential VMs on Google Cloud).
*   **Regular Security Scanning:**  Scan your deployment infrastructure and code for vulnerabilities using tools like container vulnerability scanners, static code analysis tools, and dynamic application security testing (DAST).
*   **Network Security:**
    *   Use Virtual Private Clouds (VPCs) to isolate your deployment environment.
    *   Configure firewall rules to restrict access to the deployed model from only authorized sources.
    *   Use network segmentation to further isolate different components of your application.
*   **Input Sanitization & Validation:**  Critical to prevent prompt injection or adversarial attacks.  Validate and sanitize all user inputs before feeding them to the model. Use techniques like:
    *   **Input Length Limits:** Restrict the maximum length of user inputs.
    *   **Regex Validation:**  Use regular expressions to enforce allowed character sets and data formats.
    *   **Content Filtering:** Employ content filtering APIs or libraries to detect and block malicious or inappropriate content.

**2. Secure API Access (if the model is accessed through a pre-built API):**

*   **Authentication and Authorization:**
    *   **API Keys:**  Use API keys to identify and authenticate clients accessing your API. Rotate API keys regularly.
    *   **OAuth 2.0:**  Implement OAuth 2.0 for delegated authorization.  This allows users to grant your application access to their data without sharing their credentials.
    *   **Identity-Aware Proxy (IAP):**  Use IAP to control access to your application based on user identity.  This is particularly useful for internal applications.
*   **Rate Limiting:**  Implement rate limiting to prevent abuse and denial-of-service attacks.
*   **Request Validation:**  Validate the format and content of incoming requests to prevent injection attacks.  Use a schema validator to ensure that requests conform to the expected API contract.
*   **HTTPS Enforcement:**  Always use HTTPS to encrypt communication between clients and your API.
*   **CORS Configuration:**  Properly configure Cross-Origin Resource Sharing (CORS) to restrict which domains can access your API from a web browser.

**3. Model Security:**

*   **Provenance Verification:** If possible, verify the source and integrity of the model weights. Look for signatures or checksums provided by the model developers.
*   **Adversarial Robustness Evaluation:**  Evaluate the model's robustness against adversarial attacks.  This involves testing the model's ability to resist carefully crafted inputs designed to fool it.  Tools exist to generate adversarial examples and assess model vulnerability.
*   **Regular Model Updates:**  Stay up-to-date with the latest model versions and security patches. Model developers may release updates to address security vulnerabilities or improve robustness.
*   **Monitoring and Logging:**
    *   Log all API requests and responses. This helps you detect and investigate security incidents.
    *   Monitor key metrics such as request latency, error rates, and resource utilization.
    *   Set up alerts for suspicious activity, such as unusual traffic patterns or API usage.

**4. Data Privacy and Compliance:**

*   **Data Masking and Anonymization:**  Mask or anonymize sensitive data before feeding it to the model.
*   **Data Encryption:** Encrypt data at rest and in transit.
*   **Compliance with Regulations:** Ensure that your use of the model complies with relevant data privacy regulations such as GDPR, CCPA, and HIPAA.
*   **Data Retention Policies:**  Define and enforce data retention policies to ensure that data is not stored longer than necessary.

**Industry-Standard Techniques & Tools:**

*   **Infrastructure as Code (IaC):** Use tools like Terraform or CloudFormation to define and manage your infrastructure in a declarative way. This helps you automate security configuration and enforce consistent security policies.
*   **Container Security Tools:** Use tools like Aqua Security, Twistlock, or Snyk to scan your container images for vulnerabilities.
*   **API Gateways:**  Use an API gateway (e.g., Google Cloud API Gateway, Kong, Tyk) to manage and secure your APIs. API gateways provide features such as authentication, authorization, rate limiting, and request validation.
*   **Security Information and Event Management (SIEM) Systems:**  Integrate your logs with a SIEM system (e.g., Splunk, Sumo Logic, QRadar) to provide centralized security monitoring and incident response capabilities.
*   **Penetration Testing:**  Conduct regular penetration testing to identify vulnerabilities in your application.

**Important Considerations:**

*   **Model Bias:**  Be aware of potential biases in the models you use and take steps to mitigate them. This can involve carefully selecting training data and using techniques to de-bias the model.
*   **Explainability and Transparency:**  Understand how the model works and why it makes certain predictions. This is important for building trust and ensuring that the model is not being used in a discriminatory or unethical way.
*   **Ethical Considerations:** Carefully consider the ethical implications of using the model.  Ensure that it is used responsibly and in accordance with ethical guidelines.

By implementing these security measures, you can mitigate the risks associated with using models from the Google Model Garden and ensure that your applications are secure and compliant. Remember that security is an ongoing process, so it's important to continuously monitor your systems and adapt your security measures as needed.

Gemini-2.0-Flash has done a much better job than me at explaining the model garden in a matter of seconds. Frightening.

More information here: https://pypi.org/project/google-cloud-aiplatform/