From fe933b3d28247e687aec8566986009bd82d23ab6 Mon Sep 17 00:00:00 2001 From: Quetzalli Date: Thu, 20 Aug 2026 13:32:29 +0200 Subject: [PATCH] DOC-339: Rewrite Snowflake Getting Started to mirror AWS v2 and default to lstk Mirrors the AWS Getting Started rewrite (DOC-12) information architecture: Overview, Installation, Local Development, CI Integration, AI & Agent Workflows, Auth Token, FAQ. - Overview and Installation are split into separate pages; Overview is a new SectionCards landing page, Installation leads with lstk (lstk start --type snowflake), with Docker Compose/CLI as secondary paths. - Quickstart is decoupled and removed; its content is folded into a new Local Development page with an explicit, unbroken walkthrough: start the emulator via lstk, connect the Snowflake CLI to a `localstack` connection profile, run a first query, inspect results in the Web App, then clean up. Adds a redirect from the old quickstart URL. - CI Integration is a new page with lstk-based CI snippets, and AI & Agent Workflows is a new top-level page centered on the MCP server's localstack-snowflake-client tool. - Auth Token and FAQ are restructured to match the AWS pattern while keeping Snowflake-specific content (session verify endpoint, SQL API/hostname/image-tag FAQs) that has no AWS equivalent. - Fixed three tutorial pages whose prerequisite links pointed at the now-split /snowflake/getting-started/ page. Deviates from a literal AWS mirror where the underlying product differs: no lstk wrapper for the Snowflake CLI (agents run `snow` directly against its own connection profile), no per-provider CI guide fan-out (links to the existing consolidated CI integration page instead), and no claim about Snowflake-specific LocalStack Skills coverage since it could not be verified. --- public/_redirects | 2 + .../getting-started/ai-workflows.mdx | 90 +++++++ .../snowflake/getting-started/auth-token.mdx | 209 ++++++--------- .../docs/snowflake/getting-started/ci-cd.mdx | 117 ++++++++ .../docs/snowflake/getting-started/faq.md | 20 +- .../docs/snowflake/getting-started/index.mdx | 246 ++++------------- .../getting-started/installation.mdx | 193 +++++++++++++ .../getting-started/local-development.mdx | 224 ++++++++++++++++ .../snowflake/getting-started/quickstart.md | 253 ------------------ .../aws-lambda-localstack-snowpark.md | 2 +- ...credit-scoring-with-localstack-snowpark.md | 2 +- .../s3-tables-iceberg-integration.md | 2 +- 12 files changed, 779 insertions(+), 581 deletions(-) create mode 100644 src/content/docs/snowflake/getting-started/ai-workflows.mdx create mode 100644 src/content/docs/snowflake/getting-started/ci-cd.mdx create mode 100644 src/content/docs/snowflake/getting-started/installation.mdx create mode 100644 src/content/docs/snowflake/getting-started/local-development.mdx delete mode 100644 src/content/docs/snowflake/getting-started/quickstart.md diff --git a/public/_redirects b/public/_redirects index dc5fe53e1..f5b3a0925 100644 --- a/public/_redirects +++ b/public/_redirects @@ -803,6 +803,8 @@ /snowflake/features/materalized-views/ /snowflake/features/materialized-views/ 301 /user-guide/ci /aws/ci-pipelines/ 301 /user-guide/ci/ /aws/ci-pipelines/ 301 +/snowflake/getting-started/quickstart /snowflake/getting-started/local-development/ 301 +/snowflake/getting-started/quickstart/ /snowflake/getting-started/local-development/ 301 /aws/user-guide/ci /aws/ci-pipelines/ 301 /aws/user-guide/ci/ /aws/ci-pipelines/ 301 /aws/tooling/lambda-tools/vscode-extension/ /aws/connecting/ides/vscode-extension/ 301 diff --git a/src/content/docs/snowflake/getting-started/ai-workflows.mdx b/src/content/docs/snowflake/getting-started/ai-workflows.mdx new file mode 100644 index 000000000..0475ccfbf --- /dev/null +++ b/src/content/docs/snowflake/getting-started/ai-workflows.mdx @@ -0,0 +1,90 @@ +--- +title: AI & Agent Workflows +description: Use LocalStack for Snowflake with AI coding assistants and MCP clients. +template: doc +sidebar: + order: 5 +--- + +## Introduction + +LocalStack gives AI coding assistants a local Snowflake-compatible environment to work against. Instead of letting an agent run SQL experiments against a real Snowflake account, you can ask it to create schemas, run queries, inspect results, and test data pipelines in LocalStack first. + +This is useful when you want to: + +- Prototype SQL, schemas, and data pipeline logic from natural language prompts. +- Validate AI-generated SQL, dbt models, or Snowpark code before using a real Snowflake account. +- Give an AI assistant a safe place to run queries, inspect results, and iterate on a data pipeline. + +## Common workflows + +There are two common ways to use LocalStack for Snowflake in AI-assisted development: + +- Use the [LocalStack MCP Server](/aws/developer-tools/running-localstack/mcp-server/) when your AI assistant supports MCP clients such as Cursor, Claude, Codex, or OpenCode. The server includes a dedicated Snowflake tool that runs SQL against your local emulator via the Snowflake CLI. +- Use LocalStack with the [Snowflake CLI](/snowflake/integrations/snow-cli/) directly when you want the agent to generate SQL, dbt models, or Snowpark code that you review and run locally. + +Unlike the AWS emulator, `lstk` does not proxy the Snowflake CLI the way it proxies `aws`, `terraform`, or `cdk` commands. Agent workflows that run SQL directly use the Snowflake CLI's own `localstack` connection profile, as set up in [Local Development](/snowflake/getting-started/local-development/#step-2-connect-the-snowflake-cli). + +## Quick Setup + +LocalStack provides an [`agents.md`](https://docs.localstack.cloud/agents.md) file with the full instructions your AI agent needs to get started with LocalStack, including how to configure the MCP server for the AWS, Snowflake, and Azure emulators. You can give the file directly to your agent or copy and paste the prompt below. + +```text +Fetch https://docs.localstack.cloud/agents.md and follow the instructions to set up LocalStack on my machine. +``` + +For manual setup of the MCP server, you can follow the steps below. + +## Connect an MCP client + +The LocalStack MCP Server connects MCP-compatible clients to your LocalStack environment. Once configured, your AI assistant can use LocalStack tools to start the Snowflake emulator, run SQL queries and files against it via the Snowflake CLI, inspect logs, and manage state. + +Start the MCP server with an interactive setup wizard: + +```bash +npx -y @localstack/localstack-mcp-server init +``` + +:::note +The MCP server runs locally and talks to a LocalStack instance. Your AI assistant is the MCP client. For full installation instructions, detailed setup, and the full tool reference (including the Snowflake-specific `localstack-snowflake-client` tool), see the [LocalStack MCP Server guide](/aws/developer-tools/running-localstack/mcp-server/). + +You need a valid [Auth Token](/snowflake/getting-started/auth-token/) to configure the server, and the [Snowflake CLI](https://docs.snowflake.com/en/developer-guide/snowflake-cli/index) (`snow`) installed on your `PATH` if you want the agent to run SQL directly. +::: + +## Example prompt sequence + +After LocalStack and your preferred AI tooling are configured, you can use a sequence like this: + +```text +Start the LocalStack Snowflake emulator. +``` + +```text +Create a database, schema, and table for storing customer orders, then insert a few sample rows. +``` + +```text +Run a query that summarizes total order value by customer and show me the results. +``` + +```text +Write a dbt model that reproduces this summary, and validate it against the LocalStack Snowflake emulator. +``` + +This keeps the feedback loop local while still giving the assistant a realistic Snowflake-compatible target. + +## Review before applying to Snowflake + +AI-generated SQL and data pipeline code still needs review. Treat LocalStack as the first validation step, not as a replacement for code review, tests, or production deployment controls. + +Before applying changes to a real Snowflake account, check that: + +- The generated schema and queries match your intended data model. +- Roles, warehouses, and resource names are appropriate for your project. +- Tests pass against LocalStack. +- You understand any changes the assistant made to pipeline code or configuration. + +## Next steps + +- Configure the [LocalStack MCP Server](/aws/developer-tools/running-localstack/mcp-server/) if your AI assistant supports MCP. +- Browse the [Feature Coverage](/snowflake/feature-coverage/) reference, or check the [Getting Started FAQ](/snowflake/getting-started/faq/) for common setup questions. diff --git a/src/content/docs/snowflake/getting-started/auth-token.mdx b/src/content/docs/snowflake/getting-started/auth-token.mdx index f583cf0ca..9651e3509 100644 --- a/src/content/docs/snowflake/getting-started/auth-token.mdx +++ b/src/content/docs/snowflake/getting-started/auth-token.mdx @@ -1,169 +1,111 @@ --- title: Auth Token -description: Configure your Auth Token to access and activate LocalStack for Snowflake. +description: Configure and manage your LocalStack Auth Token to activate LocalStack for Snowflake and access licensed features. template: doc sidebar: - order: 1 + order: 6 --- -import { Code, Tabs, TabItem } from '@astrojs/starlight/components'; +import { Tabs, TabItem } from '@astrojs/starlight/components'; -## Introduction +## What is an Auth Token? -An Auth Token is required to activate the LocalStack for Snowflake emulator. -It identifies and authenticates users outside the LocalStack Web Application, granting access to your workspace and to advanced features such as the Snowflake emulator image. +An Auth Token is a mandatory credential required to start the Snowflake emulator and activate licensed features. It links your running LocalStack instance to your workspace license and unlocks the services and capabilities available to your account. -Auth Tokens are issued at the workspace level in [app.localstack.cloud](https://app.localstack.cloud) and are not specific to any single LocalStack product. -The same token works across every LocalStack product your account has access to, including LocalStack for AWS, Azure, and Snowflake. +Auth Tokens are issued at the workspace level in [app.localstack.cloud](https://app.localstack.cloud) and are not specific to any single LocalStack product. The same token works across every LocalStack product your account has access to, including LocalStack for AWS, Snowflake, and Azure. -Auth Tokens come in two types: a **Developer Auth Token** and a **CI Auth Token**: +You can manage Auth Tokens from the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) in the LocalStack Web Application. -- The **Developer Auth Token** uniquely identifies a user within a workspace. - Every user has their own Auth Token. - It cannot be deleted but can be rotated for security reasons if needed. -- The **CI Auth Token** uniquely identifies a subscription rather than a specific user. - It is designed for use in CI environments and other non-developer contexts, and is stored in the workspace where it can be managed by members with appropriate permissions. - -In both cases, the Auth Token grants access to whatever product(s) the associated user or subscription is entitled to. -Both the **Developer Auth Token** and **CI Auth Token** can be managed on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens). +:::danger[Credential security] +Auth Tokens provide access to your license and workspace. Do not commit tokens to version control. If a token is exposed, rotate it immediately in the LocalStack Web Application. +::: -:::danger +## Token types -- It's crucial to keep your Auth Token confidential. - Do not include it in source code management systems, such as Git repositories. -- Be aware that if an Auth Token is committed to a public repository, it is at risk of exposure and could remain in the repository's history, even if attempts are made to rewrite it. -- In case your Auth Token is accidentally published, immediately rotate it on the [Auth Token page](https://app.localstack.cloud/workspace/auth-tokens). - ::: +| Token Type | Scope | Use Case | +| :--- | :--- | :--- | +| **Developer Token** | Individual | Local development workstations. Managed per user. | +| **CI Auth Token** | Workspace | Automated pipelines and shared runners. Managed by workspace admins. | -## Managing your License +## Managing your license -To use the LocalStack for Snowflake emulator, a license with access to Snowflake is required. -You can get a license by [signing up for a free LocalStack account](https://www.localstack.cloud/pricing) and starting a trial, or by exploring additional features with a paid offering. +To use the LocalStack for Snowflake emulator, a license with access to Snowflake is required. You can get a license by [signing up for a free LocalStack account](https://www.localstack.cloud/pricing) and starting a trial, or by exploring additional features with a paid offering. -After initiating your trial or acquiring a license, assign it to a user by following these steps: +After initiating your trial or acquiring a license, assign it to a user: -- Visit the [Users & Licenses page](https://app.localstack.cloud/workspace/members). -- Select a user in the **Workspace Members** section for license assignment. -- Define the user's role via the **Member Role** dropdown. - Single users automatically receive the **Admin** role. -- Toggle **Advanced Permissions** to set specific permissions. - Single users automatically receive full permissions. -- Click **Save** to complete the assignment. - Single users assign licenses to themselves. +1. Navigate to the [Users & Licenses page](https://app.localstack.cloud/workspace/members). +2. Identify the target user in **Workspace Members**. +3. Select the appropriate **Member Role**. +4. Save the configuration to activate the license for that identity. If you have joined a workspace, you need to be assigned a license by the workspace administrator. -When switching workspaces or licenses, make sure you are assigned to the correct license. :::note -If you do not assign a license, the Snowflake emulator will not start even if you have a valid Auth Token. +LocalStack cannot activate the Snowflake emulator unless the token belongs to a user or workspace with an assigned license, even if the Auth Token itself is valid. ::: -To view your own assigned license, visit the [My License page](https://app.localstack.cloud/workspace/my-license). -For more details on inviting users, assigning licenses, or managing roles, see [Users and Licenses](/aws/organizations-admin/managing-users-licenses/). - -## Configuring your Auth Token - -The Snowflake emulator reads the Auth Token from the `LOCALSTACK_AUTH_TOKEN` environment variable. -You can configure the Auth Token in several ways, depending on your setup. -The following sections describe the various methods of providing your Auth Token to the Snowflake container. - -:::danger +To view your own assigned license, visit the [My License page](https://app.localstack.cloud/workspace/my-license). For more details on inviting users, assigning licenses, or managing roles, see [Users and Licenses](/aws/organizations-admin/managing-users-licenses/). -- It's crucial to keep your Auth Token confidential. - Do not include it in source code management systems, such as Git repositories. -- Be aware that if an Auth Token is committed to a public repository, it is at risk of exposure and could remain in the repository's history, even if attempts are made to rewrite it. -- In case your Auth Token is accidentally published, immediately rotate it on the [Auth Token page](https://app.localstack.cloud/workspace/auth-tokens). - ::: +## Configure your token -### LocalStack CLI +Authentication requirements vary based on your chosen execution method. -You should set the `LOCALSTACK_AUTH_TOKEN` environment variable either before or during the startup of LocalStack using the `localstack` command-line interface (CLI). -When starting the Snowflake emulator, use the `--stack snowflake` flag: +### lstk - - - \nlocalstack start --stack snowflake`} - lang="shell" - /> - - - \nlocalstack start --stack snowflake`} - lang="powershell" - /> - - +The `lstk` CLI automates the authentication lifecycle. On initial execution, it triggers a browser-based OAuth flow and stores the resulting token in your system keyring. No manual environment variable configuration is required. -:::note +```bash +lstk start --type snowflake +``` -1. You can alternatively `export LOCALSTACK_AUTH_TOKEN=` in your shell session. - The CLI transmits the token to the Snowflake container, enabling license activation. -2. The `localstack auth set-token` command is only available for the `localstack` CLI and cannot be used with a Docker or Docker Compose setup. - ::: +You can alternatively set the `LOCALSTACK_AUTH_TOKEN` environment variable in your shell session; `lstk` uses it when no keyring token is present. See [Authentication](/aws/developer-tools/running-localstack/lstk/#authentication) for the full resolution order. -You have the option to run the Snowflake container in the background by appending the `-d` flag to the `localstack start` command. +### Docker and Docker Compose -### Docker +For direct container execution, inject the token as an environment variable. For complete startup examples, see the [Docker Compose](/snowflake/getting-started/installation/#docker-compose) and [Docker CLI](/snowflake/getting-started/installation/#docker-cli) installation options. -To start the Snowflake emulator via Docker, provide the Auth Token using the `-e` flag: +**Docker CLI:** -```bash {6} -docker run \ - --rm -it \ - -p 4566:4566 \ - -p 4510-4559:4510-4559 \ - -p 443:443 \ - -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:- } \ - localstack/snowflake +```bash +-e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN} ``` -For more information about starting the Snowflake emulator with Docker, take a look at our [Snowflake installation guide](/snowflake/getting-started/#alternatives). - -### Docker Compose - -To start the Snowflake emulator using `docker compose`, include the `LOCALSTACK_AUTH_TOKEN` environment variable in your `docker-compose.yml` file: +**Docker Compose:** ```yaml environment: - - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} + - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN} ``` -You can manually set the Auth Token, or use the `export` command to establish the Auth Token in your current shell session. -This ensures the Auth Token is transmitted to the Snowflake container, enabling license activation. - -### CI Environments +### CI environments -CI environments require a CI Auth Token. -Developer Auth Tokens cannot be used in CI. -CI Auth Tokens are available on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) and are configured similarly to Developer Auth Tokens. +CI environments require a CI Auth Token. Developer Auth Tokens cannot be used in CI. CI Auth Tokens are available on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) and are configured similarly to Developer Auth Tokens. -To set the CI Auth Token, add the Auth Token value in the `LOCALSTACK_AUTH_TOKEN` environment variable of your CI provider, and reference it when starting the Snowflake emulator in your CI workflow. -The same patterns used for [LocalStack in CI](/aws/ci-pipelines/) apply to Snowflake. +For complete examples, see the [CI Integration guide](/snowflake/getting-started/ci-cd/). -## Rotating the Auth Token +## Verify activation -Your personal Auth Token provides full access to your workspace and LocalStack license. -Treat it as confidential and avoid sharing or storing it in source control management systems (SCMs) like Git. +Verify the activation status by querying the Snowflake emulator's session endpoint: -If you believe your Auth Token has been compromised or becomes known to someone else, reset it without delay. -When you reset a token, the old one is immediately deactivated and can no longer access your license or workspace. -Previous tokens cannot be restored. + + -To rotate your Auth Token, go to the [Auth Token page](https://app.localstack.cloud/workspace/auth-tokens) and select the **Reset Auth Token** option. + ```bash + curl -d '{}' snowflake.localhost.localstack.cloud:4566/session + ``` -## Verifying activation + + -The simplest way to verify that the Snowflake emulator activated successfully is to query the session endpoint: + ```powershell + Invoke-WebRequest -Method POST -Body '{}' -Uri http://snowflake.localhost.localstack.cloud:4566/session + ``` -```bash -curl -d '{}' snowflake.localhost.localstack.cloud:4566/session -``` - -A successful activation returns: + + -```json -{"success": true} +```json title="Output" +{ "success": true } ``` You can also check the container logs for a message indicating successful license activation: @@ -174,14 +116,19 @@ You can also check the container logs for a message indicating successful licens Otherwise, check the [Troubleshooting](#troubleshooting) section below. +## Rotate a token + +Rotate an Auth Token if it has been exposed, shared accidentally, or stored in a place where it should not be. Go to the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) and select the reset option for the affected token. + +After rotation, update every local shell, container configuration, or CI secret that used the old token. + ## Troubleshooting -The Snowflake emulator requires a successful license activation to start. -If activation fails, the container exits and prints an error message similar to: +The Snowflake emulator requires a successful license activation to start. If activation fails, the container exits and prints an error message similar to: ```bash =============================================== -License activation failed! +License activation failed! Reason: The credentials defined in your environment are invalid. Please make sure to set the LOCALSTACK_AUTH_TOKEN variable to a valid auth token. You can find your Auth Token in the LocalStack web app https://app.localstack.cloud. @@ -192,32 +139,26 @@ The most common causes are listed below. ### Missing credentials -You need to provide an Auth Token to start the Snowflake emulator. -You can find your Auth Token on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) in the LocalStack Web Application. - -If you are using the `localstack` CLI, you can set the `LOCALSTACK_AUTH_TOKEN` environment variable to your Auth Token or use the following command to set it up: +You need to provide an Auth Token to start the Snowflake emulator. You can find your Auth Token on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) in the LocalStack Web Application. -```bash -localstack auth set-token -``` +If you are using `lstk`, run `lstk login` to authenticate through a browser-based flow, or set the `LOCALSTACK_AUTH_TOKEN` environment variable directly. ### Invalid license -The issue may occur if there is no valid license linked to your account (for example, because it has expired), or if the license has not been assigned to your user. -You can check your license status in the LocalStack Web Application on the [My License page](https://app.localstack.cloud/workspace/my-license). -If your license does not grant access to the Snowflake emulator, [contact us](https://localstack.cloud/contact/) to upgrade. +The issue may occur if there is no valid license linked to your account (for example, because it has expired), or if the license has not been assigned to your user. You can check your license status in the LocalStack Web Application on the [My License page](https://app.localstack.cloud/workspace/my-license). If your license does not grant access to the Snowflake emulator, [contact us](https://localstack.cloud/contact/) to upgrade. ### License server unreachable -LocalStack initiates offline activation when the license server is unreachable, requiring re-activation every 24 hours. -Log output may indicate issues with your machine resolving the LocalStack API domain, which can be verified using a tool like `dig`: +LocalStack initiates offline activation when the license server is unreachable, requiring re-activation every 24 hours. Log output may indicate issues with your machine resolving the LocalStack API domain, which can be verified using a tool like `dig`: ```bash dig api.localstack.cloud ``` -If the result shows a status other than `status: NOERROR`, your machine is unable to resolve this domain. -Certain corporate DNS servers may filter requests to specific domains. -Kindly reach out to your network administrator to safelist the `localstack.cloud` domain. +If the result shows a status other than `status: NOERROR`, your machine is unable to resolve this domain. Certain corporate DNS servers may filter requests to specific domains. Reach out to your network administrator to safelist the `localstack.cloud` domain. If you continue to have problems with license activation, or if the steps above do not help, do not hesitate to [contact us](https://localstack.cloud/contact/). + +## Next steps + +After configuring your Auth Token, continue to the [Local Development guide](/snowflake/getting-started/local-development/) to start the Snowflake emulator and run your first query. diff --git a/src/content/docs/snowflake/getting-started/ci-cd.mdx b/src/content/docs/snowflake/getting-started/ci-cd.mdx new file mode 100644 index 000000000..64b9d3481 --- /dev/null +++ b/src/content/docs/snowflake/getting-started/ci-cd.mdx @@ -0,0 +1,117 @@ +--- +title: CI Integration +description: Use LocalStack for Snowflake in CI pipelines to run integration tests against a local Snowflake-compatible emulator. +template: doc +sidebar: + order: 4 +--- + +import { Tabs, TabItem } from '@astrojs/starlight/components'; + +## Introduction + +LocalStack for Snowflake helps you run integration tests in CI against an emulated Snowflake instance. Your pipeline starts the Snowflake emulator inside the CI job, connects the Snowflake CLI or your test harness (dbt, Airflow, etc.), runs tests against the local endpoint, and then discards the environment when the job ends. + +## How LocalStack works in CI + +A typical CI job with LocalStack for Snowflake follows this flow: + +1. Check out your application code. +2. Start the Snowflake emulator in the CI runner. +3. Configure a CI Auth Token through the CI provider's secret manager. +4. Connect with the Snowflake CLI or your test harness. +5. Run integration tests against the LocalStack endpoint. +6. Collect logs, test reports, and artifacts from the job. + +This gives every pipeline run a fresh Snowflake-compatible environment without creating cloud resources in a real Snowflake account. + +## What changes from local development + +CI runs are usually more constrained than local development: + +- Use a dedicated **CI Auth Token** instead of a personal Developer Token. +- Store `LOCALSTACK_AUTH_TOKEN` as a protected CI secret. +- Start LocalStack non-interactively as part of the job. +- Treat the LocalStack container as ephemeral unless your workflow explicitly saves state. +- Export logs and test reports before the runner shuts down. + +Docker and Docker Compose are still common ways to run containers inside CI runners, but they are not CI tools by themselves. For container startup details, see the [Installation guide](/snowflake/getting-started/installation/#container-and-orchestration-tools). + +## Choose your CI provider + +Start with the CI system you use. These snippets show the basic emulator startup shape for each provider. + +:::note +For brevity, these snippets show only the LocalStack startup shape. They assume your CI Auth Token is already exposed to the job as the `LOCALSTACK_AUTH_TOKEN` environment variable. Store it as a secret in your CI provider before running them, and see [Authentication in CI](#authentication-in-ci) below. +::: + + + + + ```yaml + - name: Start LocalStack for Snowflake + run: | + npm install -g @localstack/lstk + lstk start --type snowflake --non-interactive + env: + LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }} + ``` + + + + + ```yaml + version: '2.1' + jobs: + localstack-snowflake-test: + machine: + image: ubuntu-2204:current + steps: + - checkout + - run: + name: Install lstk + command: npm install -g @localstack/lstk + - run: + name: Start LocalStack for Snowflake + command: lstk start --type snowflake --non-interactive + ``` + + + + + ```yaml + stages: + - test + + localstack-snowflake-test: + stage: test + image: node:20 + services: + - docker:dind + script: + - npm install -g @localstack/lstk + - lstk start --type snowflake --non-interactive + ``` + + + + +`lstk start --non-interactive` blocks until the emulator reports healthy, or exits non-zero if it fails to start within the readiness deadline (60 seconds by default). This means no separate wait step is required, unlike CI setups based on the legacy `localstack` CLI's `localstack start -d && localstack wait -t 60` pattern. Override the deadline with `--timeout` or [`LSTK_STARTUP_TIMEOUT`](/aws/developer-tools/running-localstack/lstk/#environment-variables) if your runner needs more time to pull the image. + +You can also start the emulator directly with Docker or Docker Compose inside your CI job — see the [Installation guide](/snowflake/getting-started/installation/#container-and-orchestration-tools) for the container configuration, and the existing [Continuous Integration](/snowflake/integrations/continuous-integration/) guide for complete Docker-based examples. + +## Authentication in CI + +CI environments should use a CI Auth Token. Create one from the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens), then store it as `LOCALSTACK_AUTH_TOKEN` in your CI provider's secret manager. + +Do not commit tokens to your repository or write them directly into workflow files. For more details on token types and rotation, see the [Auth Token guide](/snowflake/getting-started/auth-token/). + +## State in CI + +Most CI jobs should start with a clean LocalStack instance. A fresh instance makes test runs reproducible and avoids hidden dependencies between jobs. + +If your pipeline needs state across jobs or workflow stages, see [State Management](/snowflake/capabilities/state-management/) to save and restore named LocalStack state snapshots. + +## Next steps + +After choosing your CI provider, continue to [AI & Agent Workflows](/snowflake/getting-started/ai-workflows/) to learn how AI coding assistants can help generate, run, and test Snowflake SQL against LocalStack. diff --git a/src/content/docs/snowflake/getting-started/faq.md b/src/content/docs/snowflake/getting-started/faq.md index 5c732ba55..f05cbe0a6 100644 --- a/src/content/docs/snowflake/getting-started/faq.md +++ b/src/content/docs/snowflake/getting-started/faq.md @@ -3,7 +3,7 @@ title: FAQ description: Frequently asked questions about LocalStack for Snowflake template: doc sidebar: - order: 3 + order: 7 --- ## Core FAQs @@ -28,13 +28,25 @@ Note: In case you are deploying the LocalStack for Snowflake in a Kubernetes clu You can set the `SF_LOG=trace` environment variable in the Snowflake container to enable detailed trace logs that show all the request/response message. -When using `docker-compose` then simply add this variable to the `environment` section of the YAML configuration file. -If you're starting up via the `localstack start` CLI, then make sure to start up via the following configuration: +If you're using `lstk`, define an environment profile in your `config.toml` and reference it from your container block: + +```toml +[[containers]] +type = "snowflake" +port = "4566" +env = ["debug"] + +[env.debug] +DEBUG = "1" +SF_LOG = "trace" +``` ```bash -DOCKER_FLAGS='-e SF_LOG=trace' DEBUG=1 localstack start --stack snowflake +lstk start --type snowflake ``` +If you're using `docker-compose`, simply add these variables to the `environment` section of the YAML configuration file instead. + ### The `snowflake.localhost.localstack.cloud` hostname doesn't resolve on my machine, what can I do? On some systems, including some newer versions of MacOS, the domain name `snowflake.localhost.localstack.cloud` may not resolve properly. diff --git a/src/content/docs/snowflake/getting-started/index.mdx b/src/content/docs/snowflake/getting-started/index.mdx index 3571bf122..11979d68a 100644 --- a/src/content/docs/snowflake/getting-started/index.mdx +++ b/src/content/docs/snowflake/getting-started/index.mdx @@ -1,196 +1,68 @@ --- -title: Installation -description: Installation guide to get started with LocalStack for Snowflake. +title: Overview +description: Introduction to LocalStack for Snowflake, covering core use cases, local data pipeline development, and deployment options for development and testing. template: doc +editUrl: false sidebar: - order: 0 + order: 1 --- -import { Code, LinkButton, Tabs, TabItem } from '@astrojs/starlight/components'; -import { LOCALSTACK_AWS_VERSION } from 'astro:env/server'; - -## LocalStack CLI for Snowflake - -The easiest way to get started with LocalStack for Snowflake is by using the LocalStack CLI. It automatically pulls the correct Docker image, manages the container, and supports additional features like image updates and debugging. - -## Installing the LocalStack CLI - -The LocalStack CLI can be installed using Python, Brew (macOS), or Windows executables. - - - - - -If you prefer to install LocalStack via Python (recommended for most Snowflake users), follow the steps below. - -Please ensure the following are installed: - -- [Python](https://docs.python.org/3/using/index.html) (versions 3.7 to 3.11) -- [pip](https://pip.pypa.io/en/stable/installation/) - -Then install the LocalStack CLI: - -```bash -python3 -m pip install --upgrade localstack -``` - -:::note -To download a specific version of LocalStack, replace `` with the required version from [changelog page](/snowflake/changelog). - -```bash -python3 -m pip install localstack== -``` - -::: - -:::tip[MacOS Sierra?] -If you have problems with permissions in MacOS X Sierra, install with: - -```bash -python3 -m pip install --user localstack -``` - -::: - -:::danger -Do not use `sudo` or the `root` user when starting LocalStack. -It should be installed and started entirely under a local non-root user. -::: - - - - - -You can install the LocalStack CLI using Homebrew: - -```bash -brew install localstack/tap/localstack-cli -``` - -
-Alternative: Binary Download - -Download the binary for your architecture: - - - Intel (AMD64) - - -Then extract it: - - -
- - - - - -You can download the pre-built binary below: - - - Intel (AMD64) - - -Then extract the archive and run the binary in PowerShell. - :::note -You can download and install the Windows executable from our [GitHub Releases](https://github.com/localstack/localstack-cli/releases). +**Further reading:** Browse the [Feature Coverage](/snowflake/feature-coverage/) and [Capabilities](/snowflake/capabilities/) pages to see what LocalStack for Snowflake supports beyond this getting-started flow. ::: - - - - - -Need more options? See our [alternative installation instructions](https://docs.localstack.cloud/snowflake/getting-started/#alternatives). - -## Starting the Snowflake Emulator - -Once the CLI is installed and your [Auth Token](/snowflake/getting-started/auth-token/) is set, start the Snowflake Emulator with: - -```bash -localstack start --stack snowflake -``` - -This will pull the LocalStack for Snowfalke image and run the Docker image automatically. - -:::note -You may also start the LocalStack container using [Docker Compose](https://docs.localstack.cloud/snowflake/getting-started/#alternatives). -::: - -### Check if the emulator is running - -Run the following command: - -```bash -curl -d '{}' snowflake.localhost.localstack.cloud:4566/session -``` - -Expected response: - -```json -{ "success": true } -``` - - -## Updating LocalStack - -To keep both the CLI and emulator images up to date, run: - -```shell -localstack update all -``` - -If you want to pull a specific image tag manually, the following tags are available for the LocalStack for Snowflake Docker image: - -| Tag | Updated when | Recommended for | -|---|---|---| -| `latest` / `stable` | Tagged releases only (e.g. `2026.05.0`) | Most users — stable, release-quality builds | -| `dev` | Every merged commit on `main` | Users who need the latest unreleased changes | -| `YYYY.MM.patch` (e.g. `2026.05.0`) | Never (pinned) | Fully reproducible environments where no changes are acceptable | - -Starting with the end-of-March 2026 release, versioned Snowflake image tags follow [calendar versioning](https://calver.org/) in the `YYYY.MM.patch` format (for example, `2026.03.0`). -Refer to the available [tags on Docker Hub](https://hub.docker.com/r/localstack/snowflake/tags) for the latest releases. - - -## Alternatives - -If you prefer Docker Compose, you can start the emulator using the configuration below: - -```yaml showLineNumbers -version: '3.8' - -services: - localstack: - container_name: 'localstack-snowflake' - image: localstack/snowflake - ports: - - '127.0.0.1:4566:4566' - - '127.0.0.1:4510-4559:4510-4559' - - '127.0.0.1:443:443' - environment: - - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} - volumes: - - './volume:/var/lib/localstack' -``` - -Start the container with: - -```bash -docker-compose up -``` - -## Next steps - -Now that the Snowflake emulator is installed, you can use it for developing and testing your Snowflake data pipelines. Refer to our [Quickstart](/snowflake/getting-started/quickstart/) guide to get started. diff --git a/src/content/docs/snowflake/getting-started/installation.mdx b/src/content/docs/snowflake/getting-started/installation.mdx new file mode 100644 index 000000000..7e1da8c9a --- /dev/null +++ b/src/content/docs/snowflake/getting-started/installation.mdx @@ -0,0 +1,193 @@ +--- +title: Installation +description: Install LocalStack for Snowflake with lstk, Docker, or Docker Compose. +template: doc +sidebar: + order: 2 +--- + +import { Tabs, TabItem } from '@astrojs/starlight/components'; + +## Introduction + +LocalStack provides multiple installation paths depending on your development environment and requirements. We recommend a CLI-based installation for the most consistent local startup experience. + +Use [`lstk`](#lstk) to install, authenticate, and start LocalStack for Snowflake with minimal setup. + +LocalStack for Snowflake features require an [Auth Token](/snowflake/getting-started/auth-token/) to activate your running instance. `lstk` handles authentication through a browser-based login flow, while Docker and CI workflows can use `LOCALSTACK_AUTH_TOKEN`. + +## lstk + +`lstk` is a lightweight CLI for LocalStack that manages the authentication and container lifecycle for the AWS, Snowflake, and Azure emulators in a single workflow. + +**Requirement:** You must have a working [Docker installation](https://docs.docker.com/get-docker/) before proceeding. + +### Install lstk + + + + ```bash + brew install localstack/tap/lstk + ``` + + + ```bash + npm install -g @localstack/lstk + ``` + + + Download the binary for your platform from the [GitHub + Releases](https://github.com/localstack/lstk/releases) and add it to your + `PATH`. + + + +### Start the Snowflake emulator + +`lstk` starts the AWS emulator by default. Select the Snowflake emulator with `--type snowflake` on your first start; `lstk` records the choice in your `config.toml`, so subsequent `lstk start` calls reuse it: + +```bash +lstk start --type snowflake +``` + +The first execution initiates a browser-based login flow. Subsequent starts use credentials stored in your system keyring. + +Alternatively, run `lstk` interactively with no existing config file and choose `s` for Snowflake when prompted at the first-run emulator picker. + +To pin the emulator per-project instead of changing your global default, create a project-local `.lstk/config.toml`: + +```toml +# .lstk/config.toml +[[containers]] +type = "snowflake" +port = "4566" +``` + +### Update lstk + +```bash +lstk update +``` + +For more details, see the [lstk documentation](/aws/developer-tools/running-localstack/lstk/). + +## Container and orchestration tools + +Use these methods when you need explicit container configuration, want to run LocalStack alongside other services, or deploy LocalStack in CI environments. +For everyday local development, `lstk` is usually simpler. + +### Docker Compose + +Use Docker Compose when you want a reusable configuration file that can be shared across a team or checked into a project repository. +Create a `docker-compose.yml` with the following configuration: + +```yaml showLineNumbers +services: + localstack: + container_name: '${LOCALSTACK_DOCKER_NAME:-localstack-snowflake}' + image: localstack/snowflake + ports: + - '127.0.0.1:4566:4566' # LocalStack Gateway + - '127.0.0.1:4510-4559:4510-4559' # external services port range + - '127.0.0.1:443:443' # LocalStack HTTPS Gateway + environment: + # Activate LocalStack for Snowflake: https://docs.localstack.cloud/snowflake/getting-started/auth-token/ + - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} # required + - DEBUG=${DEBUG:-0} + - PERSISTENCE=${PERSISTENCE:-0} + volumes: + - '${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack' +``` + +Execute `docker compose up` to start. + +### Docker CLI + +Use the Docker CLI for one-off starts or when you want to test a container configuration before moving it into Compose: + +```bash +docker run \ + --rm -it \ + -p 127.0.0.1:4566:4566 \ + -p 127.0.0.1:4510-4559:4510-4559 \ + -p 127.0.0.1:443:443 \ + -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} \ + localstack/snowflake +``` + +:::note +The Docker Compose and Docker CLI examples above use the same runtime settings: + +- The `4566` port exposes the LocalStack Gateway, including the Snowflake-compatible SQL API. +- The `4510-4559` range exposes external service ports used by services that bind additional endpoints. +- The `443` port exposes the LocalStack HTTPS Gateway. +- Docker reuses a local image if one already exists. Pull explicitly or pin an image tag, such as `localstack/snowflake:`, when you need reproducible CI or team environments. +- Configuration variables can be prefixed with `LOCALSTACK_` in Docker. For instance, setting `LOCALSTACK_PERSISTENCE=1` is equivalent to `PERSISTENCE=1`. + +For more details, see the general [Docker images](/aws/customization/other-installations/docker-images/), [configuration](/aws/customization/configuration-options/), and [networking](/aws/customization/networking/) documentation, which applies across LocalStack products. +::: + +## Graphical user interfaces (GUIs) + +### LocalStack Desktop + +Manage local instances via a standalone desktop application. [Download here](https://app.localstack.cloud/download). + +## Troubleshooting + +Installation issues typically fall into one of three areas: getting your chosen install method working, activating your license, or reaching the emulator over the network. + +### lstk + +If you installed via [`lstk`](#lstk) and LocalStack fails to start, authenticate, or pull its image, see [lstk troubleshooting](/aws/developer-tools/running-localstack/lstk/#troubleshooting). + +For first-run authentication (browser login, keyring tokens, or `LOCALSTACK_AUTH_TOKEN` in CI), refer to [Authentication](/aws/developer-tools/running-localstack/lstk/#authentication) and the [Auth Token guide](/snowflake/getting-started/auth-token/). + +### Docker Compose and Docker CLI + +If you started LocalStack with [Docker Compose](#docker-compose) or the [Docker CLI](#docker-cli): + +- **License or credential errors**: see [Auth Token troubleshooting](/snowflake/getting-started/auth-token/#troubleshooting). +- **Hostname resolution or network issues**: see the [Getting Started FAQ](/snowflake/getting-started/faq/). + +Ensure you have exported `LOCALSTACK_AUTH_TOKEN` in your shell before running `docker compose up` or `docker run`. + +### View logs + +Stream container logs using the command that matches your install method: + + + + +```bash +lstk logs +``` + +For `lstk` CLI diagnostics (separate from container logs), see [Logging](/aws/developer-tools/running-localstack/lstk/#logging). + + + + +```bash +docker compose logs -f localstack +``` + + + + +```bash +docker logs -f localstack-snowflake +``` + +Use the container name from your `docker run --name` flag if you set one. + + + + +### Network connectivity + +If your Snowflake client cannot reach the emulator after installation, see the [Getting Started FAQ](/snowflake/getting-started/faq/#the-snowflakelocalhostlocalstackcloud-hostname-doesnt-resolve-on-my-machine-what-can-i-do). + +## Next steps + +Now that you've completed installation, proceed to the [Auth Token guide](/snowflake/getting-started/auth-token/) to activate LocalStack and prepare your environment for local development. diff --git a/src/content/docs/snowflake/getting-started/local-development.mdx b/src/content/docs/snowflake/getting-started/local-development.mdx new file mode 100644 index 000000000..c64fb78ab --- /dev/null +++ b/src/content/docs/snowflake/getting-started/local-development.mdx @@ -0,0 +1,224 @@ +--- +title: Local Development +description: Start LocalStack for Snowflake, connect the Snowflake CLI, and run your first query. +template: doc +sidebar: + order: 3 +--- + +## Introduction + +This guide walks you through starting the Snowflake emulator, connecting the Snowflake CLI, and running your first SQL query. You will perform the entire workflow on your local machine without a Snowflake account or consuming Snowflake credits. + +A successful walkthrough results in: + +- **A running emulator**: A local, Snowflake-compatible endpoint. +- **A Snowflake CLI connection**: A `localstack` connection profile pointed at your local emulator. +- **Sample data**: A database, schema, and table populated with sample records via a CSV upload. + +## Prerequisites + +- [Docker](https://docs.docker.com/get-docker/) engine installed and running. +- A [LocalStack account](https://app.localstack.cloud/sign-up) and a valid [LocalStack Auth Token](/snowflake/getting-started/auth-token/). +- [Snowflake CLI](https://docs.snowflake.com/en/developer-guide/snowflake-cli/installation/installation) (`snow`) installed. + +If you haven't installed LocalStack yet, follow the [installation guide](/snowflake/getting-started/installation/) to get started. + +## Step 1: Install and start the Snowflake emulator + +Start the Snowflake emulator: + +```bash +lstk start --type snowflake +``` + +The first run triggers a browser-based authentication flow. After authentication, `lstk` pulls the LocalStack for Snowflake image and initializes the container, then prints a confirmation banner once the emulator is ready. + +## Step 2: Connect the Snowflake CLI + +Create a connection profile that points the Snowflake CLI at your local emulator: + +```bash +snow connection add \ + --connection-name localstack \ + --user test \ + --password test \ + --account test \ + --host snowflake.localhost.localstack.cloud +``` + +You might be prompted for additional optional parameters, such as the connection port, database name, or warehouse name. These can be skipped. + +Test the connection: + +```bash +snow connection test --connection localstack +``` + +```bash title="Output" ++--------------------------------------------------------+ +| key | value | +|-----------------+--------------------------------------| +| Connection name | localstack | +| Status | OK | +| Host | snowflake.localhost.localstack.cloud | +| Account | test | +| User | test | ++--------------------------------------------------------+ +``` + +A `Status` of `OK` confirms the Snowflake CLI can reach your local emulator. + +:::note +This guide uses the Snowflake CLI, but LocalStack for Snowflake also works with [SnowSQL](/snowflake/integrations/snow-sql/), [DBeaver](/snowflake/integrations/dbeaver/), and the [LocalStack Web Application](/snowflake/tooling/user-interface/). See [Integrations](/snowflake/integrations/) for connection instructions for each. +::: + +## Step 3: Run your first query + +Open an interactive SQL session using the `localstack` connection profile: + +```bash +snow sql --connection localstack +``` + +In this session, we'll create a student records database that demonstrates how to create databases, schemas, and tables, upload data using a stage, and query the results. + +Create the database and use it: + +```sql +CREATE DATABASE IF NOT EXISTS STUDENT_RECORDS_DEMO; +USE DATABASE STUDENT_RECORDS_DEMO; +``` + +```bash title="Output" ++-----------------------------------------------------+ +| status | +|-----------------------------------------------------| +| Database STUDENT_RECORDS_DEMO successfully created. | ++-----------------------------------------------------+ +``` + +Create a schema and use it: + +```sql +CREATE SCHEMA IF NOT EXISTS PUBLIC; +USE SCHEMA PUBLIC; +``` + +Create the `STUDENT_DATA` table: + +```sql +CREATE OR REPLACE TABLE STUDENT_DATA ( + student_id VARCHAR(50), + first_name VARCHAR(100), + last_name VARCHAR(100), + email VARCHAR(200), + enrollment_date DATE, + gpa FLOAT, + major VARCHAR(100) +); +``` + +```bash title="Output" ++------------------------------------------+ +| status | +|------------------------------------------| +| Table STUDENT_DATA successfully created. | ++------------------------------------------+ +``` + +Create a file format and a stage for uploading files: + +```sql +CREATE OR REPLACE FILE FORMAT csv_format + TYPE = CSV + FIELD_DELIMITER = ',' + SKIP_HEADER = 1 + NULL_IF = ('NULL', 'null') + EMPTY_FIELD_AS_NULL = TRUE; + +CREATE OR REPLACE STAGE student_data_stage + FILE_FORMAT = csv_format; +``` + +Exit the SQL session (`!exit` or `Ctrl+D`), and create a `student_data.csv` file with sample records: + +```csv +student_id,first_name,last_name,email,enrollment_date,gpa,major +S001,John,Smith,john.smith@university.edu,2023-08-15,3.75,Computer Science +S002,Alice,Johnson,alice.johnson@university.edu,2023-08-15,3.92,Mathematics +S003,Bob,Williams,bob.williams@university.edu,2022-08-15,3.45,Engineering +S004,Carol,Brown,carol.brown@university.edu,2024-01-10,3.88,Physics +S005,David,Davis,david.davis@university.edu,2023-08-15,2.95,Biology +``` + +Upload the CSV file to the stage: + +```bash +snow sql --connection localstack \ + --query "PUT file://student_data.csv @student_data_stage AUTO_COMPRESS=TRUE;" +``` + +:::note +Adjust the file path to the location of your `student_data.csv` file. +::: + +```bash title="Output" +source |target |source_size|target_size|source_compression|target_compression|status |message| +----------------+-------------------+-----------+-----------+------------------+------------------+--------+-------+ +student_data.csv|student_data.csv.gz| 425| 262|NONE |GZIP |UPLOADED| | +``` + +Load the data from the stage into the table, then query it: + +```bash +snow sql --connection localstack --query " +COPY INTO STUDENT_DATA FROM @student_data_stage ON_ERROR = 'CONTINUE'; +SELECT COUNT(*) AS total_students FROM STUDENT_DATA; +" +``` + +```bash title="Output" ++----------------+ +| TOTAL_STUDENTS | +|----------------| +| 5 | ++----------------+ +``` + +The Snowflake CLI executed every statement against the locally emulated Snowflake instance. Because no actual Snowflake resources are created, you won't consume any real Snowflake credits. + +## Step 4: Inspect resources + +View the state of your local database via the [LocalStack Web Application](https://app.localstack.cloud/). Navigate to the **Snowflake** tab to inspect your running resources using the **SQL Worksheet**, which provides syntax highlighting, autocomplete, and a resource tree for your databases, schemas, and tables. + +![Running SQL queries using LocalStack Web Application](/images/snowflake/snowflake-web-ui.png) + +For more on the Web Application's Snowflake tooling, see [User Interface](/snowflake/tooling/user-interface/). + +## Step 5: Clean up + +Stop your LocalStack container to remove all emulated resources. LocalStack is ephemeral by default; stopping the instance clears the state. + +```bash +lstk stop +``` + +To persist your database, schema, and table data across restarts, see [State Management](/snowflake/capabilities/state-management/). + +Remove the local file you created in this guide: + +```bash +rm student_data.csv +``` + +## Explore more + +- **Load data from cloud storage**: Use [Storage Integrations](/snowflake/features/storage-integrations/) (currently supporting AWS S3) or a script (see [Snowflake Drivers](/snowflake/integrations/snowflake-drivers/)). +- **Automate data ingestion**: Configure [Snowpipe](/snowflake/features/snowpipe/) for automated data ingestion from external sources. +- **Use your favorite tools**: Continue developing against LocalStack for Snowflake with your preferred [integrations](/snowflake/integrations/). + +## Next steps + +You have successfully connected the Snowflake CLI and run your first query against a local Snowflake-compatible emulator. +Proceed to the [CI/CD guide](/snowflake/getting-started/ci-cd/) to learn how to integrate LocalStack for Snowflake into your automated continuous integration pipelines. diff --git a/src/content/docs/snowflake/getting-started/quickstart.md b/src/content/docs/snowflake/getting-started/quickstart.md deleted file mode 100644 index 8893f6222..000000000 --- a/src/content/docs/snowflake/getting-started/quickstart.md +++ /dev/null @@ -1,253 +0,0 @@ ---- -title: Quickstart -description: Get started with LocalStack for Snowflake in a few simple steps -template: doc -sidebar: - order: 2 ---- - -## Introduction - -This guide explains how to set up the Snowflake emulator and use Snowflake CLI to interact with Snowflake resources running on your local machine. You'll learn how to create a Snowflake database, schema, and table, upload data to a stage, and load data into the table. This quickstart is designed to help you get familiar with the Snowflake emulator and its capabilities. - -## Prerequisites - -- [LocalStack for Snowflake](/snowflake/getting-started/) -- [LocalStack CLI](/snowflake/getting-started/) -- A [LocalStack Auth Token](/snowflake/getting-started/auth-token/) -- [Snowflake CLI](/snowflake/integrations/snow-cli/) - -LocalStack for Snowflake works with popular Snowflake integrations to run your SQL queries. This guide uses the [Snowflake CLI](/snowflake/integrations/snow-cli/), but you can also use [SnowSQL](/snowflake/integrations/snow-sql/), [DBeaver](/snowflake/integrations/dbeaver/) or the [LocalStack Web Application](/snowflake/tooling/user-interface/) for this purpose. - -:::note -Each integration link includes the connection instructions needed to work with the emulator. Please be sure to follow those setup steps before running queries. -::: - -## Instructions - -Before you begin, first start the LocalStack for Snowflake emulator: - -```bash -export LOCALSTACK_AUTH_TOKEN= -localstack start --stack snowflake -``` - -In this quickstart, we'll create a student records database that demonstrates how to: - -- Create databases, schemas, and tables -- Create stages and upload data using the PUT command -- Load data from CSV files into tables -- Query your data - -### Create database, schema & table - -Create the Snowflake database named `STUDENT_RECORDS_DEMO` and use it: - -```sql -CREATE DATABASE IF NOT EXISTS STUDENT_RECORDS_DEMO; -USE DATABASE STUDENT_RECORDS_DEMO; -``` - -The output should be: - -```bash -+-----------------------------------------------------+ -| status | -|-----------------------------------------------------| -| Database STUDENT_RECORDS_DEMO successfully created. | -+-----------------------------------------------------+ -``` - -Create a Snowflake schema named `PUBLIC` and use it: - -```sql -CREATE SCHEMA IF NOT EXISTS PUBLIC; -USE SCHEMA PUBLIC; -``` - -The output should be: - -```bash -+---------------------------------------------+ -| result | -|---------------------------------------------| -| public already exists, statement succeeded. | -+---------------------------------------------+ -``` - -Last, create the table `STUDENT_DATA` in the database: - -```sql -CREATE OR REPLACE TABLE STUDENT_DATA ( - student_id VARCHAR(50), - first_name VARCHAR(100), - last_name VARCHAR(100), - email VARCHAR(200), - enrollment_date DATE, - gpa FLOAT, - major VARCHAR(100) -); -``` - -The output should be: - -```bash -+------------------------------------------+ -| status | -|------------------------------------------| -| Table STUDENT_DATA successfully created. | -+------------------------------------------+ -``` - -### Create file format & stage - -Now, create a file format for CSV files: - -```sql -CREATE OR REPLACE FILE FORMAT csv_format - TYPE = CSV - FIELD_DELIMITER = ',' - SKIP_HEADER = 1 - NULL_IF = ('NULL', 'null') - EMPTY_FIELD_AS_NULL = TRUE; -``` - -The output should be: - -```bash -+----------------------------------------------+ -| status | -|----------------------------------------------| -| File format CSV_FORMAT successfully created. | -+----------------------------------------------+ -``` - -Create a stage for uploading files: - -```sql -CREATE OR REPLACE STAGE student_data_stage - FILE_FORMAT = csv_format; -``` - -The output should be: - -```bash -+-----------------------------------------------------+ -| ?COLUMN? | -|-----------------------------------------------------| -| Stage area STUDENT_DATA_STAGE successfully created. | -+-----------------------------------------------------+ -``` - -### Upload and load sample data - -Create a new file named `student_data.csv` with sample student records: - -```csv -student_id,first_name,last_name,email,enrollment_date,gpa,major -S001,John,Smith,john.smith@university.edu,2023-08-15,3.75,Computer Science -S002,Alice,Johnson,alice.johnson@university.edu,2023-08-15,3.92,Mathematics -S003,Bob,Williams,bob.williams@university.edu,2022-08-15,3.45,Engineering -S004,Carol,Brown,carol.brown@university.edu,2024-01-10,3.88,Physics -S005,David,Davis,david.davis@university.edu,2023-08-15,2.95,Biology -``` - -Upload the CSV file to the stage using the PUT command: - -```sql -PUT file://student_data.csv @student_data_stage AUTO_COMPRESS=TRUE; -``` - -:::note -Adjust the file path to the location of your `student_data.csv` file. -::: - -The output should show the file upload status: - -```bash -source |target |source_size|target_size|source_compression|target_compression|status |message| -----------------+-------------------+-----------+-----------+------------------+------------------+--------+-------+ -student_data.csv|student_data.csv.gz| 425| 262|NONE |GZIP |UPLOADED| | -``` - -Now load the data from the stage into the table: - -```sql -COPY INTO STUDENT_DATA -FROM @student_data_stage -ON_ERROR = 'CONTINUE'; -``` - -### Verify data loading - -```sql -USE DATABASE STUDENT_RECORDS_DEMO; -USE SCHEMA PUBLIC; - -SELECT COUNT(*) as total_students FROM STUDENT_DATA; -``` - -The output should be: - -```bash -+----------------+ -| TOTAL_STUDENTS | -|----------------| -| 5 | -+----------------+ -``` - -Similarly, you can query the student details based on their GPA: - -```sql -SELECT first_name, last_name, major, gpa -FROM STUDENT_DATA -WHERE gpa >= 3.8 -ORDER BY gpa DESC; -``` - -The output should be: - -```bash -FIRST_NAME|LAST_NAME|MAJOR |GPA | -----------+---------+-----------+----+ -Alice |Johnson |Mathematics|3.92| -Carol |Brown |Physics |3.88| -``` - -Optionally, you can also query your Snowflake resources & data using the LocalStack Web Application, that provides a **Worksheet** tab to run your SQL queries. - -![Running SQL queries using LocalStack Web Application](/images/snowflake/snowflake-web-ui.png) - -### Destroy the local infrastructure - -To stop LocalStack and remove locally created resources, use: - -```bash -localstack stop -``` - -LocalStack is ephemeral and doesn't persist data across restarts. It runs inside a Docker container, and once it's stopped, all locally created resources are automatically removed. To persist the state of your LocalStack for Snowflake instance, please check out our guide on [State Management](/snowflake/capabilities/state-management/). - -## Next Steps - -Now that you've completed the quickstart, here are some additional features you can explore: - -- **Load data from cloud storage**: You can load data through our [Storage Integrations](/snowflake/features/storage-integrations/) (currently supporting AWS S3) or using a script (see [Snowflake Drivers](/snowflake/integrations/snowflake-drivers/)) -- **Automate data ingestion**: You can configure [Snowpipe](/snowflake/features/snowpipe/) for automated data ingestion from external sources -- **Use your favorite tools**: You can continue to work with your favorite tools to develop on LocalStack for Snowflake locally, see [Integrations](/snowflake/integrations/) - -## Further Reading - -You can now explore the following resources to learn more about the Snowflake emulator: - -- [Feature Coverage](/snowflake/feature-coverage/): Learn about the Snowflake emulator's features and how to use them. -- [Capabilities](/snowflake/capabilities/): Find information about the Snowflake emulator's capabilities and how to use them. - -:::note - -### How can I get help with the LocalStack for Snowflake? - -LocalStack for Snowflake is now GA. To get help, you can join the [Slack community](https://localstack.cloud/slack) and share your feedback, questions, and suggestions with the LocalStack team on the `#help` channel. If your team is using LocalStack for Snowflake, you can also request support by [contacting us](https://localstack.cloud/contact) or -[opening a GitHub Discussion](https://github.com/orgs/localstack/discussions). -::: diff --git a/src/content/docs/snowflake/tutorials/aws-lambda-localstack-snowpark.md b/src/content/docs/snowflake/tutorials/aws-lambda-localstack-snowpark.md index 368123de1..def61adcc 100644 --- a/src/content/docs/snowflake/tutorials/aws-lambda-localstack-snowpark.md +++ b/src/content/docs/snowflake/tutorials/aws-lambda-localstack-snowpark.md @@ -19,7 +19,7 @@ The code in this tutorial is available on [GitHub](https://github.com/localstack ## Prerequisites -- [`localstack` CLI](/snowflake/getting-started/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/) +- [`localstack` CLI](/snowflake/getting-started/installation/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/) - [LocalStack for Snowflake](/snowflake/getting-started/) - [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) & [`awslocal` wrapper](/aws/connecting/aws-cli/#localstack-aws-cli-awslocal) - Python 3.10 installed locally diff --git a/src/content/docs/snowflake/tutorials/credit-scoring-with-localstack-snowpark.md b/src/content/docs/snowflake/tutorials/credit-scoring-with-localstack-snowpark.md index 0957f3c8d..88aac6294 100644 --- a/src/content/docs/snowflake/tutorials/credit-scoring-with-localstack-snowpark.md +++ b/src/content/docs/snowflake/tutorials/credit-scoring-with-localstack-snowpark.md @@ -14,7 +14,7 @@ The Jupyter Notebook and the dataset used in this tutorial are available on [Git ## Prerequisites -- [`localstack` CLI](/snowflake/getting-started/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/) +- [`localstack` CLI](/snowflake/getting-started/installation/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/) - [LocalStack for Snowflake](/snowflake/getting-started/) - [Snowpark](/snowflake/integrations/snowpark) with other Python libraries - [Jupyter Notebook](https://jupyter.org/install#jupyter-notebook) diff --git a/src/content/docs/snowflake/tutorials/s3-tables-iceberg-integration.md b/src/content/docs/snowflake/tutorials/s3-tables-iceberg-integration.md index b8498733e..b1f1a9eb3 100644 --- a/src/content/docs/snowflake/tutorials/s3-tables-iceberg-integration.md +++ b/src/content/docs/snowflake/tutorials/s3-tables-iceberg-integration.md @@ -18,7 +18,7 @@ With LocalStack's Snowflake emulator, you can create catalog integrations that c ## Prerequisites -- [`localstack` CLI](/snowflake/getting-started/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/) +- [`localstack` CLI](/snowflake/getting-started/installation/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/) - [LocalStack for Snowflake](/snowflake/getting-started/) - [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) & [`awslocal` wrapper](/aws/connecting/aws-cli/#localstack-aws-cli-awslocal) - Python 3.10+ with `pyiceberg` and `pyarrow` installed