Skip to content

Pypi smoke tests should run at release #1751

Closed

Description

@jacalata
No description provided.

Activity

  1. bcantoni commented on Feb 4, 2026

    @bcantoni
    Contributor

    I asked Cursor for a proposal for how to smoke test packages before publishing (in the context of #1745) and this was the suggested approach. Not testing at all, just repeating what the AI suggested :)

    Package Testing Strategy

    This document summarizes how to test packaging changes to prevent issues like the v0.39 bug.

    The v0.39 Bug

    What happened: PR #1739 changed pyproject.toml to use where = ["tableauserverclient"] instead of where = ["."]. This caused the package to be built incorrectly with files at server/server.py instead of tableauserverclient/server/server.py, making the module unimportable.

    Why it wasn't caught: The change worked fine with editable installs (pip install -e .) but broke regular wheel installations (pip install tableauserverclient).

    Quick Local Test

    Run this script before any packaging changes:

    ./test-package.sh

    This script:

    1. Builds the wheel
    2. Inspects the wheel structure
    3. Tests installation in a clean virtual environment
    4. Verifies imports and basic functionality

    Automated Testing

    New CI Workflow

    The .github/workflows/test-package-install.yml workflow now runs on:

    • PRs that modify pyproject.toml, setup.py, or MANIFEST.in
    • PRs that modify package source code
    • Manual trigger via workflow_dispatch

    It tests wheel installation on Linux, macOS, and Windows.

    Existing Workflows

    • publish-pypi.yml: Already publishes to TestPyPI before real PyPI (good!)
    • pypi-smoke-tests.yml: Tests installation from PyPI daily (catches issues after release)

    Testing Checklist

    Before merging packaging changes:

    • Run ./test-package.sh locally
    • Verify CI passes (test-package-install workflow)
    • Inspect wheel contents: unzip -l dist/*.whl | grep tableauserverclient
    • For major changes: test via TestPyPI
    • For releases: wait for pypi-smoke-tests to pass

    Key Differences: Editable vs Wheel Install

    Aspect Editable (pip install -e .) Wheel (pip install pkg.whl)
    How it works Symlinks to source directory Copies files to site-packages
    Tests packaging ❌ No ✅ Yes
    Good for Development Testing releases
    Catches structure bugs ❌ No ✅ Yes

    Always test with wheel installs before releasing!

    Resources

    • Full guide: docs/testing-package-builds.md
    • Test script: ./test-package.sh
    • CI workflow: .github/workflows/test-package-install.yml

    Quick Reference

    # Build and test locally
    ./test-package.sh
    
    # Manually inspect wheel
    python -m build
    unzip -l dist/*.whl | grep tableauserverclient
    
    # Test in clean environment
    python -m venv test-env
    source test-env/bin/activate
    pip install dist/*.whl
    cd /tmp
    python -c "import tableauserverclient as TSC; print('Success!')"
    deactivate
    
    # Test via TestPyPI (after pushing to GitHub)
    git tag v0.39.1-test
    git push origin v0.39.1-test
    # Wait for workflow, then:
    pip install --index-url https://test.pypi.org/simple/ \
                --extra-index-url https://pypi.org/simple/ \
                tableauserverclient
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions