You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Redact password column from CSV import logging and error output - #1862
Fixes two paths in UserItem.CSVImport that could leak the password column from a user-import CSV file:
_validate_import_line_or_throw logged logger.info(f"Reading user {line[:4]}"). The intent was to avoid printing the password, but slicing four characters only obscures usernames shorter than four characters -- longer usernames leak part of the row, and if the layout ever changed the password itself could reach the log.
validate_file_for_import returned raw lines in its invalid_lines list when a row failed validation. Any caller that logged or surfaced those lines would expose the password unmasked.
The per-column debug log inside _validate_import_line_or_throw printed the PASS column value verbatim.
Row-level logs now include only the username (parsed by partition on the first comma) and are demoted to DEBUG.
invalid_lines entries pass through a new _redact_password_column helper that replaces column 1 with ***, preserving the line's original line ending.
The per-column DEBUG log masks the PASS column with ***.
The redaction helper uses the same naive str.split(",") parsing already used throughout the file. A password value containing commas is misaligned across columns; only the fragment landing in column 1 is masked. A test documents this limitation -- a proper CSV parser would be a larger change and is out of scope here.
Test plan
Four new tests in test/test_user_model.py covering: DEBUG log on a valid row (positive-and-negative assertion so a fix that only deletes the log line would fail), invalid-row path (both the returned invalid_lines and the logger output), embedded-comma limitation, and unit-level edges on _redact_password_column (LF, CRLF, no newline, empty password, trailing comma, single column).
`UserItem.CSVImport.validate_file_for_import` and
`_validate_import_line_or_throw` wrote the raw CSV line -- including
the password column -- to any caller-supplied logger at INFO/DEBUG
level, and the whole raw line was pushed into the `invalid_lines`
list returned to callers when a row failed validation. Anyone using
the sample logger config or forwarding logs to a centralized system
would see clear-text passwords in the log stream.
Changes:
- `validate_file_for_import` logs only the username (column 0) at
DEBUG, and calls a new `_redact_password_column` helper before
appending an invalid row to the returned list.
- `_validate_import_line_or_throw` masks the PASS column value as
`***` before logging it. Other column values still logged as-is
for debugging.
- Both callers changed from INFO to DEBUG for these per-row messages;
large imports were spamming operator-visible logs.
- Two regression tests capture logs and returned invalid_lines to
assert the secret never appears in either place, plus a positive
assertion that a `***` masked value IS logged so a future refactor
that just removes the log line entirely doesn't pass.
Fixes#1829.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR hardens UserItem.CSVImport to prevent accidental disclosure of the CSV password column via logging and error/invalid-line output, addressing security issue #1829.
Changes:
Restrict row-level import logs to username-only and emit them at DEBUG level.
Redact the password column (PASS) when returning invalid_lines, and mask the PASS value in per-column DEBUG logs.
Add regression/unit tests covering log redaction behavior and helper edge cases; document the embedded-comma limitation; update changelog.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
File
Description
tableauserverclient/models/user_item.py
Adds password redaction helper and updates CSV import logging/invalid-line handling to avoid password disclosure.
test/test_user_model.py
Adds regression tests ensuring passwords do not appear in DEBUG logs or invalid row output; adds unit tests for redaction helper behavior.
CHANGELOG.md
Documents the security fix for CSV import password logging/output behavior (Fixes #1829).
The original _redact_password_column stripped only \n before splitting
on commas, so on a CRLF-terminated 2-column line ("user,pass\r\n") the
\r rode with the password field, was thrown away when that field became
"***", and the returned line silently downgraded to LF. Now the exact
trailing sequence (\r\n, \n, or none) is preserved and reattached after
the redaction.
Adds a regression test for the CRLF-with-password-as-last-field case.
Also corrects the CHANGELOG: the pre-fix code logged the first four
characters of each raw row at INFO (not DEBUG), which is why the
downgrade to DEBUG matters in addition to the redaction.
_redact_password_column now preserves the exact trailing sequence (\r\n, \n, or none) after redaction. Split-and-rejoin no longer downgrades CRLF to LF when the password is the last field.
Regression test "user,pass\r\n" -> "user,***\r\n" added to test_redact_password_column_helper.
CHANGELOG updated separately: the pre-fix code logged the first four characters of each raw row at INFO, not DEBUG, so the entry now describes both the row-level INFO→DEBUG downgrade and the password-column redaction.
Inadvertently pulled the password-column mask
(safe_value = "***" if column == PASS else value) into this PR while
restructuring the validation loop for the warn-on-unknown-auth
change. The mask is #1862's core content and shouldn't sneak in
through the site-role decompose PR -- reviewers on either PR would
see mysterious overlap. #1862's full feature (log mask + INFO->DEBUG
downgrade + _redact_password_column helper for invalid_lines
sanitization) stays where it belongs, on jac/csv-import-privacy.
The `column = ColumnType(i)` local rename stays because it's used
by the log line's `{column.name}` format and by the AUTH branch's
comparison; that's plain cleanup and doesn't overlap with #1862.
… _evaluate_site_role (#1812)
* fix: CSVImport AUTH column, case-insensitive validation, username preservation; add find_by_name
Fixes for UserItem.CSVImport (issue #1809):
- MAX=8 (was 7=AUTH index): 8-column lines with auth type no longer
rejected as "too many columns"
- create_user_from_line no longer lowercases the whole line before
splitting — username case is preserved
- _validate_import_line_or_throw normalizes license/admin/publisher to
lowercase and auth to canonical form before comparison, so 'Viewer',
'Creator', 'SAML', 'tableauidwithmfa' etc. are all accepted
- Add TableauIDWithMFA to valid auth values in validation (was missing)
- 5 new tests covering each fix
Add QuerysetEndpoint.find_by_name(name) (issue #1810):
- Thin wrapper over .filter(name=name) returning a list
- Available on all content-item endpoints (workbooks, datasources,
views, users, projects, groups)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: add auth column to create_users_csv, fix create_from_file extension check and debug prints
- create_users_csv was producing 7-column CSV, silently dropping
auth_setting; bulk_add roundtrip would lose auth type
- create_from_file extension check used filepath.find("csv") which
evaluates as falsy only when "csv" is at index 0, letting all other
paths through; fixed to "csv" not in filepath
- remove two debug print() calls left in create_from_file
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor: extract _decompose_site_role into CSVImport, symmetrical with _evaluate_site_role
Moves the ad-hoc site role → (license, admin_level, publish) logic from
create_users_csv into UserItem.CSVImport._decompose_site_role, making it
the explicit inverse of _evaluate_site_role.
Also fixes pre-existing bugs in the decomposition:
- ExplorerCanPublish was emitted as license="ExplorerCanPublish" (not a
valid CSV license value); now correctly "Explorer" with publish=1
- SiteAdministrator (legacy role) was emitting license="" via
str.replace; now maps to ("Explorer", "Site", "1")
- Non-admin roles now emit admin_level="None" (explicit CSV spec value)
rather than "" (empty string); both are accepted by the server but
"None" is consistent with the spec and _evaluate_site_role input
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: decompose unsupported site roles to license="Invalid" not "Unlicensed"
Legacy values in UserItem.Roles (UnlicensedWithPublish, ViewerWithPublish,
Guest, SupportUser) have never been accepted by the server-side CSV license
parser (workgroup: CsvLicenseRoleTypeConverter). The initial _decompose_site_role
default of ("Unlicensed", "None", "0") silently coerced these to a valid but
semantically wrong Unlicensed user, replacing an old server-side per-row
rejection with silent success.
Default to license="Invalid" instead so the server continues to reject rows
with USER_CSV_INVALID_LICENSE, preserving the pre-refactor observable
behavior for callers who inspect job results for per-row failures. Batch
resilience is unaffected: bad rows fail, good rows succeed, no client-side
throw takes down the whole bulk_add call.
Follow-up: deprecate UnlicensedWithPublish/ViewerWithPublish from
UserItem.Roles (never worked on any code path); see forthcoming issue.
* docs: clarify _decompose_site_role's asymmetric legacy-role handling
The role map handles legacy UserItem.Roles values in two ways depending
on whether the server has a modern equivalent. SiteAdministrator,
Publisher, Interactor, and ReadOnly map to their current-model
equivalents; UnlicensedWithPublish, ViewerWithPublish, Guest, and
SupportUser fall to license="Invalid" so the server rejects the row.
Reviewers otherwise read the code and see "why does Publisher get
mapped but Guest doesn't?" - the answer is server behavior, not
arbitrary choice. Docstring now says so.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: accept "1"/"0" and "true"/"false" for publisher, add round-trip test
Two related changes to make `_evaluate_site_role` and
`_decompose_site_role` a real inverse pair:
- `_evaluate_site_role` now accepts "1", "true", and "yes" for the
publisher column (and, symmetrically, treats anything else as "no").
Previously only "yes" was accepted, which meant `_decompose_site_role`
emitting "1" for Creator/ExplorerCanPublish would round-trip as
Explorer. The set of publisher values mirrors what
`_valid_attributes[publisher]` already documents as legal
(["yes", "true", "1", "no", "false", "0"]), so this brings the two
code paths in agreement.
- Added a parametrized `test_decompose_then_evaluate_round_trips`
covering every entry in `_role_map` plus the two documented label
asymmetries (ServerAdministrator -> SiteAdministrator on the way
back; legacy roles Publisher/Interactor/ReadOnly/SiteAdministrator
folded into their modern equivalents). If either function drifts,
a specific input names the broken case rather than the whole loop
dying on the first mismatch.
- `_decompose_site_role` docstring updated with a Round-trip note
naming the two intentional asymmetries so readers do not have to
reason about them from the mapping table.
Full suite still passes: 884 passed, 1 skipped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Address fresh-eyes review nits before un-drafting
- Strip five non-ASCII characters (arrow + em-dashes) from docstrings
and comments; project convention is ASCII-only.
- create_user_from_line now passes an unknown auth value through
unchanged (previously silently .get'd to None). _validate_import_
line_or_throw on the same input raises, so the two entry points
no longer disagree on what counts as valid.
Not addressed here (per fresh-eyes review, non-blocking):
- Deprecated create_from_file's "csv" substring check remains loose
(matches report.csv.bak, rejects USERS.CSV); pre-existing on a
@deprecated method, out of scope for this refactor.
- PR body test-plan checkbox that's actually complete: tick before
un-drafting.
59 user tests pass.
* Warn (don't raise) on unknown auth values during CSV import
Post-merge follow-up on #1811 -- the strict behavior it introduced
(create_user_from_line raises ValueError on any auth value not in
_AUTH_CANONICAL) would block CSV imports against newer servers as
soon as Tableau ships an auth type TSC's hardcoded list doesn't yet
know about. This is the same category of stale-list problem the
_set_values enum-guard bypass exists to avoid on the server-parse
path.
Both entry points now warn and pass the value through:
- create_user_from_line: unknown values raise a UserWarning naming
the value and known set, then get assigned to auth_setting as-is.
If the value really is a typo, the server rejects the row when
the request posts -- a slightly-later error, but the import stays
possible against forward-compatible servers.
- _validate_import_line_or_throw: same shape. Skips the allowlist
check for the AUTH column when the value isn't in _AUTH_CANONICAL,
so validate_file_for_import doesn't return the row as invalid.
Server-version-aware validation would be the cleaner long-term fix
here (and for the enum-guard bypass in _set_values); noted for
planning, not filing an issue.
Tests updated: two former "raises ValueError" cases now assert
pytest.warns(UserWarning) and confirm the raw value round-trips
onto UserItem.auth_setting.
* Strip password-mask line: it belongs to #1862, not this PR
Inadvertently pulled the password-column mask
(safe_value = "***" if column == PASS else value) into this PR while
restructuring the validation loop for the warn-on-unknown-auth
change. The mask is #1862's core content and shouldn't sneak in
through the site-role decompose PR -- reviewers on either PR would
see mysterious overlap. #1862's full feature (log mask + INFO->DEBUG
downgrade + _redact_password_column helper for invalid_lines
sanitization) stays where it belongs, on jac/csv-import-privacy.
The `column = ColumnType(i)` local rename stays because it's used
by the log line's `{column.name}` format and by the AUTH branch's
comparison; that's plain cleanup and doesn't overlap with #1862.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Belt-and-braces: today PASS has an empty allowlist so the validator returns
early, but if a future PR adds password-format checks (length, complexity,
banned-chars) the raw password would leak via ValueError("Invalid value
{item} for {column_type}"). Skip the call entirely for PASS so the invariant
doesn't rely on the allowlist staying empty.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_validate_import_line_or_throw assigned `column` twice in the loop body
(leftover from the redaction diff). Also had its own inline PASS-boundary
check (`column.value < PASS.value`) duplicating the same boundary logic
_redact_password_column derives independently via `ColumnType.PASS.value`.
Factor both into one _is_sensitive_column helper so there's a single
source of truth for "is this column sensitive enough to mask."
This only masks the second parsed CSV field. A syntactically valid but malformed import row with an unquoted comma in its password, such as u,secret,tail,foo,site,yes,email, is returned as u,***,tail,..., so invalid_lines still exposes a password fragment (and later validation errors can expose shifted fragments too). Since this helper is used only for rejected rows, ambiguous rows should be replaced with a safe placeholder rather than returning the remaining parsed fields.
Review effort: Lite
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two paths in
UserItem.CSVImportthat could leak the password column from a user-import CSV file:_validate_import_line_or_throwloggedlogger.info(f"Reading user {line[:4]}"). The intent was to avoid printing the password, but slicing four characters only obscures usernames shorter than four characters -- longer usernames leak part of the row, and if the layout ever changed the password itself could reach the log.validate_file_for_importreturned raw lines in itsinvalid_lineslist when a row failed validation. Any caller that logged or surfaced those lines would expose the password unmasked._validate_import_line_or_throwprinted the PASS column value verbatim.Fixes #1829.
What changes
invalid_linesentries pass through a new_redact_password_columnhelper that replaces column 1 with***, preserving the line's original line ending.***.The redaction helper uses the same naive
str.split(",")parsing already used throughout the file. A password value containing commas is misaligned across columns; only the fragment landing in column 1 is masked. A test documents this limitation -- a proper CSV parser would be a larger change and is out of scope here.Test plan
test/test_user_model.pycovering: DEBUG log on a valid row (positive-and-negative assertion so a fix that only deletes the log line would fail), invalid-row path (both the returnedinvalid_linesand the logger output), embedded-comma limitation, and unit-level edges on_redact_password_column(LF, CRLF, no newline, empty password, trailing comma, single column).🤖 Generated with Claude Code