In the hyper-accelerated world of AI-assisted engineering, developers face a critical dilemma: intelligence vs. privacy. How do you leverage the elite reasoning of flagship models like Claude without surrendering your proprietary source code to the cloud?

This guide resolves that tension by building a Hybrid AI Workflow. We are merging two powerhouse ecosystems:

1. Claude Code: An agentic CLI that provides deep reasoning, multi-step planning, and autonomous file manipulation.
2. Ollama: A local model engine that ensures your most sensitive code never leaves your workstation.

By using Claude for high-level architectural guidance and Ollama for local execution and private validation, we create a development loop that is both immensely powerful and completely secure. This post details the exact setup to make this synergy work seamlessly.

Understanding Claude Code

![[Pasted image 20260412070456.png]]

To make the concept easier to grasp, think of this setup as a car. In this analogy, Claude Code is the cars body and dashboard, while the LLM is the engine.

The "body" (Claude Code) provides the interface, the tools, and the logic for interacting with your files and terminal. But without an "engine" (the LLM) to provide the reasoning power, the car won't move. You can swap the engine depending on your needs:

• Local LLMs (via Ollama): Like a custom-built engine in your own garage. It gives you complete control and privacy, running entirely on your local hardware.
• Anthropics Native Models: Like a high-performance engine from a professional racing team. You access it via API (Claude 3.5/4.x models), offering peak intelligence but requiring an external connection.
• Third-Party Hosts: Like a leased engine from a service provider, offering flexibility across different providers.

What makes Claude Code unique is its agentic workflow. It doesn't just answer questions; it can "see" your codebase, execute commands, and iterate on code until a task is complete. By pairing this sophisticated "body" with a local "engine" via Ollama, you get the best of both worlds: high-level autonomy with local privacy.

Strategic Tradeoffs: Finding Your Ideal AI Workflow

Before diving into the setup, it is crucial to understand that integrating these two systems involves several architectural tradeoffs. No single solution is perfect; the right choice depends entirely on your project's constraints.

🛡 1. Data Privacy and Security (The Biggest Tradeoff)

• Cloud LLMs (e.g., Claude): Offer best-in-class performance and massive model sizes, but your prompts and context are processed on a third-party server. This is unsuitable for handling highly sensitive, regulated, or proprietary data.
• Local LLMs (via Ollama): Offer maximum privacy. Since the model runs entirely on your machine, data never leaves your local environment. This is mandatory for compliance (HIPAA, GDPR, etc.) or working with unreleased IP. However, the model performance ceiling is dictated by your hardware.

🛠 2. Tooling Depth and Integration Complexity

• Cloud LLMs (Claude Code): Are designed to be 'agentic' within a structured toolset. They are excellent at understanding multi-step development tasks, calling external APIs, and managing complex project logic because the tool definitions are part of their prompt context.
• Local LLMs (Ollama): While Ollama supports running various models, making them reliably execute complex, multi-step, system-level tooling (like a full compiler/interpreter chain) often requires more explicit, prompt-level instruction engineering or building custom wrappers, as they don't inherently possess the same high-level agentic framework that Claude's tool-use system provides out-of-the-box.

🚀 3. Model Performance vs. Resource Requirements

• Cloud LLMs: Access to the absolute cutting edge (e.g., Opus 4.6) with minimal local overhead.
• Local LLMs: The quality is gated by the model size and your hardware (RAM, GPU VRAM). You might have to settle for smaller, more efficient models (like CodeLlama or Phi-3) that still perform well enough for the task, even if they don't match the flagship cloud models' peak capability.

Summary: * 1. Local LLM (Ollama): * The Pro: Absolute data privacy and zero usage costs. * The Con: Performance is strictly limited by your local hardware (GPU/RAM) and the size of the model your machine can handle.

* 2. Third-Party Hosting APIs (e.g., OpenRouter, Groq): * The Pro: Generally more cost-efficient than native APIs while offering high speeds. * The Con: While providers offer privacy terms, data still leaves your network to an external server. Performance can be inconsistent depending on the provider's current traffic and hardware.

* 3. Native LLM APIs (Anthropic/Claude): * The Pro: The "Gold Standard." You get the highest reasoning capabilities, the fastest response times, and the largest context windows available. * The Con: This is the most expensive option and carries the highest data privacy risk, as your prompts are processed directly on the vendor's environment.

Step 1: Install Ollama (The Local Engine)

Ollama is an indispensable tool that simplifies running and managing open-source LLMs locally. It abstracts away the complexity of running various models (Llama, Mistral, etc.) into a single, easy-to-use command-line interface.

Prerequisites: Ollama is required to serve as the local backend for models we want to use in conjunction with Claude Code.

To install Ollama, run the official script in your terminal:

curl -fsSL https://ollama.com/install.sh | sh

This command downloads and executes the installer, setting up the necessary services and making the ollama command available in your shell.

Increase the context memory size

If you are using a Modelfile, ensure you set the num_ctx parameter to at least 32768 (32k) or higher, depending on your hardware VRAM. This ensures that when Claude Code queries your local model, it has enough room to "see" the entire relevant code block in a single pass.

How to optimise for code:

![[Pasted image 20260411215635.png]]

When working with code, "Context is King." A standard 2,048 or 4,096 token window is rarely enough to hold a modern projects file structure, let alone the contents of multiple source files and their dependencies.

To prevent your local engine from "forgetting" the beginning of your file as it reads the end, you must manually increase the context window in Ollama. This allows the model to maintain a larger "active memory" of your codebase.

Install Claude Code

curl -fsSL https://claude.ai/install.sh | bash

Post downloading Claude Code open it with Ollama rather than opening in directly from the termial using the below command. 

Open Claude code using Ollama :

ollama launch claude

![[Pasted image 20260411220132.png]]

Alternatives for Claude Code

Here are the best alternatives in 2026, grouped by type. Many developers mix tools (e.g., an IDE for daily work + a terminal agent for heavy lifting). Top contenders often include Cursor, OpenAI's Codex (or Codex CLI), GitHub Copilot, and open-source options like Cline or Aider, pi agent etc.

Conclusion

The synergy between Claude Code and Ollama represents a significant shift in AI-assisted development. We are moving away from simple "chat with your code" interfaces toward autonomous, local-first development loops.

By following this setup, youve unlocked a workflow that respects your data privacy without sacrificing the sophisticated reasoning of an agentic CLI. Whether you choose to use local models for sensitive internal tools or switch to cloud models for complex architectural refactors, you now have the ultimate hybrid development environment.

As local models continue to close the gap with their cloud counterparts, this local-first approach will likely become the standard for professional software engineering. Happy coding!

👋 Introduction If you're a developer, cloud architect, or AI enthusiast who liked llama OCR, this post is for you. In this tutorial, you'll build a simple, llama OCR (Optical Character Recognition)** web app that:

• Uses OCI Generative AIs Vision LLMs for meta

• Extracts structured text from images (like receipts, scanned forms)

• Runs locally on your machine with Streamlit

• Doesnt require any front-end coding

🎯 Who Is This For?

This app is ideal for:

• Oracle Cloud (OCI) users exploring GenAI Vision models

• Developers who want to prototype document AI tools

• Data teams that frequently handle scanned invoices, receipts, or forms

• Anyone looking to integrate LLM + OCR in a no-code UI

What You'll Build

A web UI that allows you to:

• Upload an image (receipt, invoice, screenshot) in the application.

• Get the extracted Markdown output from the image using LLM

• View and copy the structured text

🏛 Why OCI Gen AI service?

While there are many cloud providers offering Generative AI services, heres why Oracle Cloud Infrastructure (OCI) stands out for this project:

🔒 Enterprise-Grade Security

Oracle offers built-in data residency, encryption, and compartment isolation, making it ideal for industries handling sensitive scanned documents like finance, healthcare, or government.

💰 Cost-Effective GenAI

OCIs flexible pricing and pay-as-you-go options for Generative AI inference make it more affordable than comparable solutions on other hyperscalers.

Native LLM Support

OCI hosts open-source LLMs like Meta LLaMA 3, Mistral, and Command R directly in your region. This means lower latency and no external API keys or SaaS contracts needed.

🧱 Full Stack Integration

From Object Storage to Data Science, Monitoring, and Vault, everything plugs in natively. You can integrate OCR output with downstream analytics, automation, or data pipelines with ease.

🔧 Developer Friendly

• One config file (~/.oci/config)

• No API gateway hoops

• Full Python SDK and CLI support

• IAM-based access control If youre already on OCI, this setup takes you from raw image to structured text in minutes.

🛠 Prerequisites

• ~OCI CLI configured~ (~/.oci/config)

• Access to OCI Generative AI Service in of the regions mentioned below

• A Vision-capable model deployed (like meta.llama-3.2-90b-vision-instruct)

• Python 3.8+

• Install required Python packages

Regions with Generative AI

💻 Setting Up a Virtual Environment (Windows & macOS/Linux)

Creating a virtual environment helps isolate dependencies and ensures your Streamlit OCR app doesnt interfere with other Python projects on your system.

For Windows:

Open Command Prompt (cmd) or PowerShell:

#Navigate to your project folder
cd path\\to\\your\\project

#Create a virtual environment
python -m venv venv

#Activate the virtual environment
venv\\Scripts\\activate

#Install dependencies
pip install streamlit oci

For macOS/Linux:

Open Terminal:

#Navigate to your project directory
cd ~/path/to/your/project

#Create a virtual environment
python3 -m venv venv

# Activate the virtual environment
source venv/bin/activate

# Install dependencies
pip install streamlit oci

Once you're done, launch the app with:

streamlit run ocr_vision_app.py

Find the full code in the github : mukundmurali-mm/llama-ocr-oci: Use llama OCR for extracting texts using OCI Gen AI services

Model context protocol (MCP)

• It is a protocol that allows LLMs to access custom tools and services.

• If your LLM wants to talk to an agent for a specific task then with MCP it is easy to integrate the LLM to this Agent.

Why MCP is needed ?

MCP solves the following challenges in the current way of connecting LLMs to AI Agents:

• Vendor lock-in: LLMs are tied to specific AI Agent providers, making it difficult to switch providers.

• Security: LLMs need to access data and services which can lead to security issues.

• Integration complexity: LLMs need to integrate with different AI Agents, which can be complex and time-consuming.

General Architecture

• MCP Hosts: Programs like Claude Desktop, IDEs, or AI tools that want to access data through MCP

• MCP Clients: Protocol clients that maintain 1:1 connections with servers

• MCP Servers: Lightweight programs that each expose specific capabilities through the standardized Model Context Protocol

• Local Data Sources: Your computers files, databases, and services that MCP servers can securely access

• Remote Services: External systems available over the internet (e.g., through APIs) that MCP servers can connect to

MCP Example :

🧠 MCP Data Interface for OCI Object Storage

This project is a Model Context Protocol (MCP)-based API interface for Oracle Cloud Infrastructure (OCI) Object Storage. It allows AI agents, tools like Ollama, and natural language interfaces to programmatically list and read objects from OCI buckets via structured JSON requests.

📌 Features

• List objects in an OCI bucket

• Read and return file contents from a bucket

• Minimal and easy-to-extend MCP-style API

• FastAPI-powered Python server

• CLI-driven backend using oci os commands

📦 Use Cases

This MCP server is ideal for:

• AI/LLM agents needing data from OCI buckets

• Streamlit or LangChain apps accessing cloud storage

• FinOps or Security tools fetching bucket inventories

• Conversational UIs querying cloud resources

• Cloud-native notebooks fetching .csv, .json, etc. for ML

Read more and get the source code from my Github repo :

MCP

In today's rapidly evolving technological landscape, artificial intelligence has become an integral part of how we conduct research and analyze information. This simple solution helps you setup a research assistant chatbot which will help you to converse with your document.

Project Overview

Oracle Cloud Infrastructure's AI Agents provide robust, scalable, and secure AI capabilities that can transform how businesses conduct research and analyze information. Our application leverages these capabilities through a sleek, user-friendly interface built with Streamlit, making advanced AI interactions accessible to everyone.

Pre-requisites

• Create the OCI AI Agent endpoints.

• Create the data source with Bucket with All objects option selected.

• Create the Bucket in the same compartment as OCI AI Agent.

• IAM policies for authorization - https://docs.oracle.com/en-us/iaas/Content/generative-ai-agents/iam-policies.htm

Key Features

• Seamless Authentication: The application integrates smoothly with OCI's authentication system, using your existing OCI CLI config credentials for secure access.

• Interactive Chat Interface: Built with Streamlit, the application provides a responsive and intuitive chat experience that feels natural and engaging.

• Session Persistence: Your chat history is maintained throughout your session, allowing for contextual conversations and easy reference to previous interactions.

• Enterprise-Ready: With built-in error handling and user feedback mechanisms, the application is designed for reliability and professional use.

• Data Privacy: All Files are securely stored in OCI Object Storage Bucket.

Limitation

• Currently OCI AI Agents supports only TXT and PDF format files.

• The maximum size for a file to be ingested is 100 MB.

• Your tenancy must have the US Midwest (Chicago) region. Generative AI Agents is only available in this region.

• Your OCI AI Agents and Buckets should be in the same compartment(This is a limitation for this application alone and not a general OCI AI Agent limitation).

Technical Implementation

The application is built using Python and leverages several key technologies:

• Streamlit: For creating a responsive and modern web interface

• OCI Python SDK: For seamless integration with Oracle Cloud Infrastructure

• Python 3.7+: Ensuring compatibility with modern Python features

Getting Started

Setting up the application is straightforward:

1. Create and activate a Python virtual environment:

python -m venv .venv 
source .venv/bin/activate  # On macOS/Linux

or

.venv\Scripts\activate  # On Windows

2. Install the required dependencies using pip:

pip install -r requirements.txt

3. Configure your OCI credentials in the OCI CLI config profile (~/.oci/config)

4. Save the below code inside the virtual env(ex: app.py) and run the application

Streamlit run app.py

5. Pass in your required parameters in the application and start chatting.

P.S: whenever there is a new object added or exiting object is deleted to the bucket which needs to be part of this Application then run 'create Injection job' once the object is uploaded/deleted.

Code

import streamlit as st
import oci
import os
import time
# Set page configuration with OCI favicon
st.set_page_config(
    page_title="OCI AI Research Assistant",
    page_icon="https://www.oracle.com/favicon.ico",
    layout="wide"
)
from oci.config import from_file
from oci.generative_ai_agent_runtime.generative_ai_agent_runtime_client import GenerativeAiAgentRuntimeClient
from oci.generative_ai_agent_runtime.models.chat_details import ChatDetails
from oci.generative_ai_agent import GenerativeAiAgentClient
from oci.generative_ai_agent.models import CreateDataIngestionJobDetails
from oci.object_storage import ObjectStorageClient
from oci.object_storage.models import CreateBucketDetails
from tempfile import NamedTemporaryFile
def initialize_oci_clients(profile_name="DEFAULT", agent_endpoint_id=None):
    """Initialize OCI clients with the specified profile and create a session"""
    try:
        st.write(f"Attempting to load config profile: {profile_name}")
        config = from_file(profile_name=profile_name)
        st.write("Config loaded successfully")
        st.write(f"Using region: {config.get('region')}")

        # Use the appropriate service endpoint based on your region
        service_endpoint = "https://agent-runtime.generativeai.us-chicago-1.oci.oraclecloud.com"
        st.write(f"Using service endpoint: {service_endpoint}")

        # Initialize GenAI client with service endpoint
        genai_client = GenerativeAiAgentRuntimeClient(
            config,
            service_endpoint=service_endpoint
        )

        # Create a session if agent_endpoint_id is provided and session doesn't exist
        if agent_endpoint_id and 'chat_session_id' not in st.session_state:
            try:
                create_session_response = genai_client.create_session(
                    create_session_details=oci.generative_ai_agent_runtime.models.CreateSessionDetails(
                        display_name="USER_Session",
                        description="User Session"),
                    agent_endpoint_id=agent_endpoint_id)
                st.session_state.chat_session_id = create_session_response.data.id
                st.write("Chat session created successfully")
            except Exception as e:
                st.error(f"Error creating chat session: {str(e)}")

        # Initialize Object Storage client
        object_storage_client = ObjectStorageClient(config)

        # Initialize Identity client
        identity_client = oci.identity.IdentityClient(config)

        st.write("OCI clients initialized")
        return genai_client, object_storage_client, identity_client, config
    except Exception as e:
        st.error(f"Error initializing OCI clients: {str(e)}")
        st.error("Please check if your OCI config file (~/.oci/config) exists and contains the correct profile")
        return None, None, None, None
def list_objects(object_storage_client, namespace, bucket_name):
    """List objects in a bucket"""
    try:
        # Send the request to service with minimal required parameters
        list_objects_response = object_storage_client.list_objects(
            namespace_name=namespace,
            bucket_name=bucket_name,
            fields="name,size,timeCreated"  # Only fetch essential fields
        )
        return list_objects_response.data.objects
    except Exception as e:
        st.error(f"Error listing objects: {str(e)}")
        return []
def upload_file(object_storage_client, namespace, bucket_name, file):
    """Upload a file to object storage"""
    try:
        # Read the file content
        file_content = file.read()

        # Upload the file to Object Storage using put_object
        put_object_response = object_storage_client.put_object(
            namespace_name=namespace,
            bucket_name=bucket_name,
            object_name=file.name,
            put_object_body=file_content,
            content_type=file.type if hasattr(file, 'type') else None
        )
        return True
    except Exception as e:
        st.error(f"Error uploading file: {str(e)}")
        return False
def delete_object(object_storage_client, namespace, bucket_name, object_name):
    """Delete an object from object storage"""
    try:
        # Send the delete request to service
        object_storage_client.delete_object(
            namespace_name=namespace,
            bucket_name=bucket_name,
            object_name=object_name
        )
        # Verify deletion by trying to get object metadata
        try:
            object_storage_client.head_object(
                namespace_name=namespace,
                bucket_name=bucket_name,
                object_name=object_name
            )
            st.error(f"Failed to delete {object_name}. Object still exists.")
            return False
        except:
            # If we get an error trying to get the object, it means it's deleted
            return True
    except Exception as e:
        st.error(f"Error deleting object: {str(e)}")
        return False
def list_data_sources(profile_name, compartment_id):
    """List available data sources"""
    try:
        # Initialize the GenerativeAiAgent client
        config = from_file(profile_name=profile_name)
        generative_ai_agent_client = GenerativeAiAgentClient(config)

        # List data sources
        response = generative_ai_agent_client.list_data_sources(
            compartment_id=compartment_id,
            lifecycle_state="ACTIVE"
        )

        return response.data.items if response.data else []
    except Exception as e:
        st.error(f"Error listing data sources: {str(e)}")
        return []
def create_ingestion_job(profile_name, compartment_id, data_source_id):
    """Create a data ingestion job"""
    try:
        # Initialize the GenerativeAiAgent client
        config = from_file(profile_name=profile_name)
        generative_ai_agent_client = GenerativeAiAgentClient(config)

        # Create the ingestion job
        response = generative_ai_agent_client.create_data_ingestion_job(
            create_data_ingestion_job_details=CreateDataIngestionJobDetails(
                compartment_id=compartment_id,
                data_source_id=data_source_id,
                display_name=f"Ingestion-Job-{int(time.time())}",  # Unique name using timestamp
                description="Data ingestion job created from Research Assistant"
            )
        )

        return response.data
    except Exception as e:
        st.error(f"Error creating ingestion job: {str(e)}")
        return None
def get_chat_response(client, agent_endpoint_id, message):
    """Get response from the chat agent"""
    try:
        # Validate agent endpoint ID
        if not agent_endpoint_id or not agent_endpoint_id.strip():
            st.error("Agent Endpoint ID is required")
            return None

        # Ensure we have a session ID
        if 'chat_session_id' not in st.session_state:
            # Create a new session if we don't have one
            try:
                create_session_response = client.create_session(
                    create_session_details=oci.generative_ai_agent_runtime.models.CreateSessionDetails(
                        display_name="USER_Session",
                        description="User Session"),
                    agent_endpoint_id=agent_endpoint_id)
                st.session_state.chat_session_id = create_session_response.data.id
            except Exception as e:
                st.error(f"Error creating chat session: {str(e)}")
                return None

        # Send the chat request
        response = client.chat(
            agent_endpoint_id=agent_endpoint_id,
            chat_details=ChatDetails(
                user_message=message,
                should_stream=False,  # Set to False for now until we implement streaming properly
                session_id=st.session_state.chat_session_id
            )
        )

        # Debug: Print response structure
        st.write("Response data attributes:", dir(response.data))

        # Return the response - accessing the correct attribute
        return response.data.message
    except Exception as e:
        st.error(f"Error getting chat response: {str(e)}")
        return None
def main():
    """Main function for the OCI AI Research Assistant application"""
    st.title("OCI AI Research Assistant")

    # Configuration Section in Sidebar
    with st.sidebar:
        st.header("Configuration")

        # Display available profiles from ~/.oci/config
        config_file = os.path.expanduser("~/.oci/config")
        available_profiles = []

        if os.path.exists(config_file):
            with open(config_file, 'r') as f:
                content = f.read()
                profiles = [line.strip('[').strip(']') for line in content.split('\n') if line.strip().startswith('[')]
                available_profiles = profiles

        #st.write("Available profiles:", ", ".join(available_profiles))

        # OCI Configuration
        profile_name = st.selectbox(
            "OCI Profile Name",
            options=available_profiles,
            index=available_profiles.index("DEFAULT") if "DEFAULT" in available_profiles else 0
        )
        agent_endpoint_id = st.text_input("Agent Endpoint ID")
        compartment_id = st.text_input("Compartment ID")

        # Object Storage Configuration
        namespace = st.text_input("Namespace Name")
        bucket_name = st.text_input("Bucket Name")

        # Initialize button
        if st.button("Initialize Clients"):
            # Validate required inputs
            if not agent_endpoint_id or not agent_endpoint_id.strip():
                st.error("Agent Endpoint ID is required")
                return

            # Store all inputs in session state
            st.session_state.profile_name = profile_name
            st.session_state.agent_endpoint_id = agent_endpoint_id
            st.session_state.compartment_id = compartment_id
            st.session_state.namespace = namespace
            st.session_state.bucket_name = bucket_name

            # Initialize OCI clients with agent endpoint ID
            genai_client, object_storage_client, identity_client, config = initialize_oci_clients(
                profile_name=profile_name,
                agent_endpoint_id=agent_endpoint_id
            )

            if all([genai_client, object_storage_client, identity_client]):
                st.session_state.genai_client = genai_client
                st.session_state.object_storage_client = object_storage_client
                st.session_state.identity_client = identity_client
                st.session_state.config = config
                st.success("OCI clients and chat session initialized successfully!")

    # Main content area for chat
    if hasattr(st.session_state, 'genai_client'):
        st.markdown("""
        Welcome to your AI Research Assistant! Ask any question, and I'll help you find the information you need.
        """)

        # Initialize chat history if it doesn't exist
        if 'messages' not in st.session_state:
            st.session_state.messages = []
        # Display chat history
        for message in st.session_state.messages:
            with st.chat_message(message["role"]):
                st.markdown(message["content"])
        # Chat input
        if prompt := st.chat_input("What would you like to research?"):
            # Add user message to chat history
            st.session_state.messages.append({"role": "user", "content": prompt})

            # Display user message
            with st.chat_message("user"):
                st.markdown(prompt)

            # Get AI response
            with st.chat_message("assistant"):
                try:
                    response = get_chat_response(
                        st.session_state.genai_client,
                        st.session_state.agent_endpoint_id,
                        prompt
                    )
                    if response:
                        st.markdown(response)
                        # Add assistant response to chat history
                        st.session_state.messages.append({"role": "assistant", "content": response})
                except Exception as e:
                    st.error(f"Error getting response: {str(e)}")

        # Object Storage in Sidebar
        if hasattr(st.session_state, 'object_storage_client'):
            with st.sidebar:
                st.markdown("---")

                # Ingestion Job Section
                st.header("Data Ingestion")

                # List data sources
                data_sources = list_data_sources(
                    st.session_state.profile_name,
                    st.session_state.compartment_id
                )

                if data_sources:
                    # Create a list of data source names and IDs for the selectbox
                    data_source_options = {f"{ds.display_name} ({ds.id})": ds.id for ds in data_sources}
                    selected_source = st.selectbox(
                        "Select Data Source",
                        options=list(data_source_options.keys())
                    )

                    if st.button("Create Ingestion Job", type="primary"):
                        with st.spinner("Creating ingestion job..."):
                            result = create_ingestion_job(
                                st.session_state.profile_name,
                                st.session_state.compartment_id,
                                data_source_options[selected_source]
                            )
                            if result:
                                st.success(f"Created ingestion job: {result.id}")
                else:
                    st.warning("No active data sources found in the compartment.")

                st.markdown("---")
                st.header("Object Storage")

                # Upload section
                st.subheader("Upload File")
                uploaded_file = st.file_uploader("Choose a file to upload", key="sidebar_uploader")
                if uploaded_file is not None:
                    if st.button("Upload"):
                        if upload_file(st.session_state.object_storage_client, 
                                     st.session_state.namespace, 
                                     st.session_state.bucket_name, 
                                     uploaded_file):
                            st.success(f"File {uploaded_file.name} uploaded successfully!")
                            st.experimental_rerun()

                # List Objects section
                st.subheader("Objects in Bucket")

                # Get the current list of objects
                objects = list_objects(
                    st.session_state.object_storage_client,
                    st.session_state.namespace,
                    st.session_state.bucket_name
                )

                # Add refresh button
                if st.button("Refresh Objects", type="primary"):
                    st.rerun()

                if objects:
                    for obj in objects:
                        col1, col2 = st.columns([3, 1])
                        with col1:
                            st.write(f"📄 {obj.name}\n{obj.size:,} bytes")
                        with col2:
                            delete_button_key = f"delete_{obj.name}"
                            if st.button("🗑", key=delete_button_key, help=f"Delete {obj.name}"):
                                try:
                                    with st.spinner(f"Deleting {obj.name}..."):
                                        if delete_object(
                                            st.session_state.object_storage_client,
                                            st.session_state.namespace,
                                            st.session_state.bucket_name,
                                            obj.name
                                        ):
                                            st.success(f"Deleted {obj.name}")
                                            st.rerun()
                                        else:
                                            st.error(f"Failed to delete {obj.name}")
                                except Exception as e:
                                    st.error(f"Error deleting {obj.name}: {str(e)}")
                        st.divider()
                else:
                    st.info("No objects found in this bucket")

                    st.markdown(response)
                    # Add assistant response to chat history
                    st.session_state.messages.append({"role": "assistant", "content": response})
if __name__ == "__main__":
    main()

Conclusion

The OCI AI Research Assistant showcases how enterprise-grade AI can be made accessible and user-friendly. Whether you're conducting academic research, analyzing complex data, or building knowledge management solutions, this project provides a solid foundation for your AI-powered research endeavors.

New features will be added to this application in the future.

Introduction

This blog post explores Steampipe and Powerpipe, powerful tools for gaining insights into your cloud resources, particularly for Oracle Cloud Infrastructure (OCI). We'll cover the basics of Steampipe, Powerpipe, and their parent company Turbot, the supported cloud providers, and delve specifically into how these tools can be used with OCI and Terraform.

What is Steampipe?

Steampipe is an open-source tool that enables users to query cloud APIs, code repositories, and other services using SQL. It acts as a universal translator, eliminating the need for specialized knowledge of various platforms and allowing you to leverage your existing SQL skills to interrogate your infrastructure. Steampipe essentially turns your cloud resources into a database that can be queried, analysed, and reported on.

What is Powerpipe?

Powerpipe builds upon Steampipe's functionality by providing a visual interface for creating dashboards and reports. It takes the raw data gathered by Steampipe and transforms it into easily understandable visualisations, making it easier to identify trends, potential security risks, and cost optimization opportunities. Powerpipe is particularly beneficial for DevOps, SecOps, and FinOps teams as it provides a consolidated view of cloud resources and their configurations.

What is Turbot?

Steampipe and Powerpipe are products of Turbot, a company that provides cloud governance and automation solutions. Turbot offers both open-source and commercial products based on Steampipe and Powerpipe technology.

Cloud Providers Supported by Steampipe

Steampipe has a wide range of plugins that connect to various cloud providers and services. Some of the major supported cloud providers include:

• Alibaba Cloud

• Amazon Web Services (AWS)

• Azure

• DigitalOcean

• Google Cloud Platform (GCP)

• Heroku

• IBM Cloud

• Linode

• Oracle Cloud Infrastructure (OCI)

• Scaleway

• Snowflake

Focusing on OCI: Steampipe, Powerpipe, and Terraform

For Oracle Cloud Infrastructure (OCI), Steampipe provides comprehensive coverage through its plugin. This plugin allows users to query various OCI services and resources using SQL. In addition to basic querying, Steampipe for OCI can be used for:

• Compliance Checks: The oci_compliance mod allows you to run security and compliance checks against your OCI resources. It supports benchmarks like CIS, helping you identify misconfigurations and potential vulnerabilities.

  

  

• Insights and Reporting: The oci_insights mod provides pre-built dashboards for visualizing your OCI resources. This helps gain a better understanding of your cloud infrastructure and its configuration.

• Cost Optimization: The oci_thrifty mod focuses on finding unused and underutilized resources. This helps optimize your cloud spending and reduce costs.

• Terraform is a popular Infrastructure as Code (IaC) tool used to manage and provision cloud resources. The terraform_oci_compliance mod allows you to scan your Terraform code, plans, and state files for potential security misconfigurations before deploying to OCI.

• 

Examples and Use Cases for OCI

Let's illustrate the power of Steampipe with some practical examples for OCI:

• Querying OCI Users: You can retrieve details about your OCI users with a simple SQL query like:

    select name, id, is_mfa_activated from oci_identity_user;
  

  This query retrieves the name, ID, and MFA activation status of all OCI users.

• Checking for Publicly Accessible Buckets: To enhance security, you can identify object storage buckets that are publicly accessible using a query like:

    select name from oci_objectstorage_bucket where public_access_type != 'NoPublicAccess';
  

  This query lists the names of all buckets that allow some form of public access.

• Identifying Unused Compute Instances: For cost optimization, you can find compute instances that are potentially unused with a query like:

    select display_name, shape, time_created
    from oci_core_instance
    where state = 'Stopped'
    order by time_created desc;
  

  This query lists stopped instances, ordered by their creation time, allowing you to investigate instances that have been stopped for a long time.

  OCI Insights Dashboard Questions

  While the sources don't explicitly list the questions that OCI Insights dashboards answer, they do provide clues about their capabilities. Based on the information in the sources, here's a breakdown of potential questions these dashboards can help answer:

  Resource Inventory and Properties:

  • How many resources do I have? This basic question can be answered across various resource types, providing a quick overview of your OCI footprint.

  • How old are my resources? Understanding the age of resources can be helpful for lifecycle management and identifying potential candidates for decommissioning or upgrades.

Security Posture:

• Are there any publicly accessible resources? This is a critical security question, as publicly accessible resources can pose significant risks. OCI Insights dashboards can likely help identify such resources, allowing you to take corrective actions.

• Is encryption enabled, and what keys are used for encryption? These dashboards can likely provide insights into the encryption status of resources, helping you assess your data protection measures.

Data Management:

• Is versioning enabled? This is particularly relevant for services like object storage, where versioning provides data recovery and protection against accidental deletions. OCI Insights dashboards can potentially show the versioning status of your resources.

Other Potential Insights:

The sources mention that OCI Insights dashboards are available for 10+ services, including Block Storage, Compute, Identity, Object Storage, VCN, and more. Given this breadth of coverage, it's likely that these dashboards can also help answer questions related to:

• Resource utilization and performance: Dashboards could potentially show metrics related to CPU utilization, storage usage, network traffic, and other performance indicators.

• Cost allocation and trends: Insights into resource usage can be further used to analyse cost allocation and identify potential areas for optimisation.

• Compliance with specific standards: While the sources primarily focus on CIS benchmarks, it's possible that OCI Insights dashboards could also address compliance requirements for other standards, depending on the available mods and configurations.

Conclusion and Next Steps

Steampipe and Powerpipe are valuable tools for anyone working with cloud infrastructure, especially OCI. They provide a unified and intuitive way to query, analyZe, and visualize your cloud resources using the familiar SQL language. By leveraging these tools, you can gain a deeper understanding of your cloud infrastructure, improve security and compliance, and optimize costs.

To learn more about Steampipe and Powerpipe:

• Visit the Steampipe Hub: Explore the various Steampipe plugins and their documentation.

• Visit the Powerpipe Hub: Discover the available Powerpipe mods and see how they can be used to create insightful dashboards.

• Explore Turbot's Website: Find information about Turbot's products and solutions built on Steampipe and Powerpipe technology.

Features and Capabilities:

• Source Grounded AI: Unlike tools like Perplexity that rely on searching the open web, NotebookLM works by uploading your own documents, limiting its knowledge to the information you provide. This eliminates the risk of hallucinations, ensuring that the AI only generates responses grounded in your sources.

• Multimodal Support: NotebookLM can handle a variety of file formats, including Google Docs, Google Slides, PDFs, images, and even YouTube videos and audio files. This versatility makes it ideal for research projects involving diverse types of information.

• Extensive Context Window: It supports up to 50 sources with up to 200,000 words per file, allowing you to build a knowledge base of up to 4 million words. This is much larger than the context windows of many other AI tools, enabling more comprehensive analysis.

• Audio Overview Generation: One of the standout features is its ability to summarize your sources into a podcast-style discussion between two AI voices. This can provide an engaging and efficient way to consume large amounts of information, though it's important to note that this feature is more prone to hallucinations.

• Precise Citations: NotebookLM cites its sources with pinpoint accuracy, highlighting the specific text it's drawing from. This is particularly useful for academic research or any situation where verifying information is crucial.

Use Cases with Examples:

• Keeping Up with Niche Information: Upload weekly transcriptions of YouTube videos or podcasts in your industry to get a quick summary and analysis.

• Client Relationship Management: Store transcripts of client calls to quickly recall past discussions, identify recurring themes, and gain insights into the relationship.

• Market and Competitive Research: Analyze competitor websites and marketing materials to uncover their messaging, target audience, and potential gaps in their approach.

• Content Research:

  • YouTube Content Research: Analyze top-performing YouTube videos on a topic to identify common themes, effective hooks, and unique angles.

  • Content Optimization: Analyze the top-ranking articles for a target keyword to identify main topics covered, semantic keywords, and opportunities to make your content more comprehensive and relevant.

• Turning a Blog Article into a Podcast: Upload a blog article, generate an audio overview, and edit the transcript to create a podcast script in your own voice.

• Audience Research: Analyze LinkedIn comments on posts by influential figures in your niche to understand audience desires, pain points, and potential content ideas.

• Academic Paper Summarization: Upload a complex academic paper and ask the AI to summarize key findings, explain concepts in simpler terms, or extract specific information.

• Meeting Notes Analysis: Upload meeting transcripts to quickly extract key points, identify action items, and generate summaries or proposals.

• Writing Assistance:

  • Idea Generation: Use NotebookLM to brainstorm related ideas or avenues of research based on your existing notes.

  • Outlining and Structuring: Generate outlines, key points, and suggestions for improving your writing.

  • Drafting and Rewriting: Ask the AI to rewrite your drafts based on specific instructions or sources.

Limitations:

• Beta Stage: Being in beta, NotebookLM can have occasional bugs and limitations.

• Limited Source Integration: Currently, it only supports uploading Google Docs and PDFs. You can't directly connect to other note-taking apps.

• Source Limit: The maximum of 20 sources might require you to consolidate documents or manually copy and paste text if you have a large number of sources.

• Challenges with Specific Tasks: Like many AI tools, NotebookLM can struggle with complex math, poorly formatted PDFs, and retrieving highly specific details.

Data Privacy:

• Google Terms of Service Apply: Your use of NotebookLM is governed by Google's Terms of Service.

• No Personal Data Used for Training: Google emphasizes that your personal data is never used to train NotebookLM.

• Feedback Review for Consumer Accounts: If you choose to provide feedback while logged in with a consumer Google account, human reviewers may access your queries, uploads, and the AI's responses.

• Privacy for Workspace Users: For Google Workspace or Google Workspace for Education users, your interactions with NotebookLM are not reviewed by humans and are not used for AI model training.

Conclusion:

NotebookLM presents a powerful new way to research, learn, and create using your own knowledge base. By combining the power of large language models with a focus on source grounded AI, it offers a level of control and accuracy not found in traditional search engines or AI chatbots. While still in its early stages, NotebookLM has the potential to revolutionize how we work with information.

Contrasting NotebookLM with ChatGPT and Perplexity

Here's a comparison of NotebookLM with ChatGPT like LLMs and Perplexity like answer engines, highlighting their key differences:

NotebookLM

• Focus: Acts as a personal AI research assistant, enabling users to interact with their uploaded documents.

• AI Model: Powered by Google's Gemini 1.5 Pro.

• Data Source: Utilizes a user-defined knowledge base, limited to the information within uploaded documents (up to 50 sources with a maximum of 4 million words).

• Key Features:

  • Supports diverse file formats like Google Docs, PDFs, images, YouTube videos, and audio files.

  • Generates podcast-style audio summaries of source materials.

  • Provides precise citations, linking responses back to the specific text in the uploaded documents.

• Strengths:

  • Source-grounded responses: Eliminates hallucinations by restricting the AI to user-provided information.

  • Large context window: Facilitates comprehensive analysis by accommodating a vast knowledge base.

  • Privacy-focused: Personal data is not used for training, and options exist for limiting human review of interactions.

• Limitations:

  • Restricted to Google Docs and PDF uploads, lacking direct integration with other note-taking apps.

  • Limited to 20 sources, requiring manual consolidation for larger datasets.

  • May face challenges with specific tasks like complex math or poorly formatted PDFs.

ChatGPT

• Focus: Functions as a general-purpose conversational AI chatbot, capable of engaging in open-ended conversations and creative tasks.

• AI Model: Powered by OpenAI's large language models (e.g., GPT-3.5, GPT-4).

• Data Source: Trained on a massive dataset of text and code scraped from the internet, enabling it to generate human-like text on a wide range of topics.

• Key Features:

  • Excels in creative writing, text summarization, translation, code generation, and answering general knowledge questions.

• Strengths:

  • Vast knowledge base: Access to a broad range of information allows for responses on diverse subjects.

  • Creative capabilities: Generative abilities make it suitable for tasks requiring imagination and language fluency.

• Limitations:

  • Prone to hallucinations: Can generate inaccurate or fabricated information due to its reliance on internet data.

  • Limited control over sources: Users cannot restrict the AI to specific sources, making fact-checking more challenging.

Perplexity

• Focus: Designed as an AI-powered search engine that provides answers to user queries alongside citations.

• AI Model: Leverages large language models for understanding and responding to queries.

• Data Source: Retrieves information from the internet, relying on its web search capabilities to answer questions.

• Key Features:

  • Presents answers in a concise and informative manner, often including bullet points and key takeaways.

  • Cites sources to provide transparency and allow for verification.

• Strengths:

  • Real-time information: Accesses up-to-date information from the web.

  • Focus on source attribution: Emphasizes transparency by providing links to the sources used in its responses.

• Limitations:

  • Potential for bias: Results can be influenced by the ranking algorithms of search engines.

  • Less control over knowledge scope: Users have less control over the specific sources the AI utilizes compared to NotebookLM.

Key Differences in a Nutshell:

In summary, NotebookLM distinguishes itself through its focus on source-grounded AI, enabling users to have focused conversations with their own knowledge base. This offers a level of accuracy and privacy not found in ChatGPT or Perplexity, making it a valuable tool for researchers, writers, and anyone seeking to leverage AI with their personal or professional data.

The problem in this scenario is, For monitoring all the north-south outbound traffic i.e. the egress internet traffic from the instances has to be sent to the Palo Alto so the ingress traffic cannot be directly to the Load balancer via the internet gateway because that would be asymmetric routing as the return traffic will be through Palo Alto. So the ingress traffic has to come through Untrust Vnic of the Palo Alto as that is the exit interface as well.

Architecture

NAT Configuration inside Palo Alto

You can configure a direct D-NAT for this configuration to translate the untrust IP to the load balancer IP but the Palo alto just drops this traffic at the untrust VNIC itself and doesn't capture this traffic. The reason being forward session is not getting created. The solution for this is to create a SNAT with the Load balancer as the source and the untrust NIC as NAT IP with Bi-Directional checkbox checked.

Palo Alto Configuration

Create the NAT policy :

1. Select Policies > NAT and click Add

2. On the General tab, enter a descriptive Name for the NAT rule.

3. On the Original Packet tab, select the zone you created for your DMZ in the Source Zone section (click Add and then select the zone) and the zone you created for the external network from the Destination Zone list.

4. In the Source Address section, Add the address object you created for your internal web server address.

5. On the Translated Packet tab, select Static IP from the Translation Type list in the Source Address Translation section and then select the address object you created for your external web server address from the Translated Address list.

6. In the Bi-directional field, select Yes.

7. Click OK.

8. Click Commit.

This type of architecture is not limited to OCI and can be applied to other cloud services as well.

We already know what ChatGPT or Google's Bard is and how it helps us. In a nutshell, we give an input to these AI models and it gives us an corresponding output. This output can be relatively correct or incorrect to some instances. This depends upon the input we give and this is called the prompt.

The process of creating an efficient prompt is called as prompt engineering.

We will see what are the different types of Prompt Engineering techniques that we can leverage.

Prompt Principles

There are 2 principles which we need to take into consideration while working on it.

1. Write clear and specific instructions.

   • Using Delimiters : Prompting is just like asking a person something, we need to be clear and specific in what we want. In Prompt Engineering we are going to specify this with the help of delimiters (""", ```, < >, <tag></tag>) which are basically used to separate the section inside prompt. Delimiters are also used to prevent any kind of prompt injection.

   • Ask for a structured output.

   • Check if conditions are met.

   • Few short prompting, by giving few examples.

2. Give the model time to think.

   • Specify the set of steps to complete a task.

   • Instruct the model to work out its own solution before rushing to conclusion.

Hallucination

At times there will be scenario where the AI model will give incorrect answers but it never knows that its a wrong answer and the model assumes that this is the correct answer for the prompt given. This is often referred to as Hallucinations in AI.

Iterative Process

• Give the idea as prompt.

• Understand the output that is generated from that prompt.

• Analyze if the desired output is generated or not.

• Make the necessary changes in the prompt and re-input it as new prompt.

Summarizing Text

Summarizing text prompt is generally used to brief a large set of text in few words to give its complete context. We can summarize a text with respect to specific context as well which later can be used as a email's body. for example: if you get a summary of a overall shipping operation then you can summarize the data with specific key values like the dates mentioned in it.

Inferring learning

Its a task where the model takes the input text and analyze it by extracting label and values using prompt engineering.

Sentiment Analysis

Sentiment analysis is a type of Inferring where you can take a text and understand on what level of emotion that text was written. This is generally used in product review analysis.

Zero-Shot Learning

In short, Zero-shot learning algorithm is getting an output without giving any example of labelled data to the prompt.

Expanding

This is right opposite to what summarize text does, you give a set of instructions or steps to the AI model and the model will generate a large set of text for it. Generally used in generating email messages.

Chatbot

The most exciting use case of the LLM model is to use it to build a customer chatbot and make it act as a customer service agent to answer the questions of user input.

This is achieved with the help of a helper function which says the AI model to have a conversational interaction with the user to store the user inputs in its memory.

Prompt engineering when used properly will help you extracting the best potential from LLM model and should be used in a responsible way which help in solving real time problems.