Repository navigation
feat: add the postcode grammar, data, vectors and standards - #1
Merged
Merged
Conversation
Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
- Remove REUSE.toml __pycache__ annotation (ignored by .gitignore) - Split ruff.toml select list to 100 column limit - Add Python.md Ruff settings: preview rules, extend-select, C90, T10 - Add mccabe max-complexity and pylint max-nested-blocks settings - Update per-file-ignores to scripts/tests/** with PT rule excluded - Rename generic test variables for clarity: data → precision_rules, value → document, PRECISIONS → SEGMENT_NAMES - Fix line-too-long in test by extracting intermediate variable - Fix collection-literal-concatenation using unpacking syntax Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
TELL-6 asks for names that say what a value holds. DATA held the path of the data folder, and thresholds held precision names, not limits. ruff.toml now also says why the tests turn off the pytest-style rules. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
Linters miss some of the agent tells in the coding standards, such as long lines in config files, debug calls in shipped code, emoji and files named utils. check-tells finds them in every tracked file, and make check now runs it. The issue template wraps two long lines, and the TELL-1 text now names the exemption for JSON data files, so the repo passes its own check. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
PY-11 asks for an Example section in each public docstring, and such examples often call print. TELL-13 now skips doctest lines, which start with >>> or ..., in the same way as it skips comment lines. The standards now say so, and PY-11 asks for doctest prompts. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
check-tells missed Swift NSLog calls. It also skipped the debug, to-do and line-length checks on code files in a folder named messages or locales. Line numbers were wrong after comment lines in commit messages, and splitlines hid U+2028 and U+2029. Doctest prompts now count only in Python, the to-do check covers config files, skip markers count only in code and config, and two more markers flag generated files. New tests pin each path rule and each debug pattern. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
A code file in a locales or i18n folder is code, so check-tells holds it to the ASCII rule, and translations live in data files. The standards now say so, and say that skip comments work in code and config files only. Two tests pin the catalogue rule and the line split in commit messages. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
Every SDK must give the same result for the same input, so the cases live in one place. build_vectors.py writes them from the data files, and make check fails when a vector file is out of date. The separators now include the soft hyphen and the two direction marks, because the grammar removes hyphens and zero-width characters. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The builder numbers each case by its position, so a new case moves the ids after it. Each SDK runs the vectors of one spec version, so an id and that version name one case. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
A wrong SDK could pass every vector. No parse case held a byte order mark, 13 separators had no case, no case showed that a suggestion keeps the district, and no case held non-ASCII digits or partial codes of 5, 8 or 10 characters. The builder now writes one case for each separator in data/format.json. A test ties the GPS limits in data/precision.json to the vectors, and descriptions are unique across all files. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The GPS test passed even without the cases just above each limit, and no test tied the separator list to its cases. Both tests now fail when a case goes missing. A new case shows that stateName does not trim spaces, and the grammar says that legacy codes use ASCII digits. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The checks in make check help only if every pull request runs them. CI now runs them on Python 3.11, the oldest version that the scripts support, and gitleaks scans the full history for secrets. Each action is pinned to a commit SHA (SEC-6). Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
A moved image tag could swap the scanner for one that always passes, so the secrets job now pins the image by digest. The check job also runs on the newest Python, because the README promises 3.11 or later. The uv cache is off, because the repo has no lock file to key it. SEC-5 now names the triggers that CI uses: each pull request and each push to main. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
SEC-5 now says that CI runs on each pull request and each push to main. The language standards still said each push. They now use the same words as SEC-5. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
NFKC can take seconds on a long run of combining marks, and parse normalises its whole input before it checks the length. A server that parses untrusted text could stall. parse now rejects input with more than 64 code points as bad_length before it normalises, and isLegacy returns false for it. The limit lives in data/format.json, and vectors pin both sides of it. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
A grapheme counter or an SDK that runs NFKC before it counts passed every vector and still left the stall open. New cases count combining marks one by one, show that the count comes before NFKC, and catch a limit check that ignores a final line feed. The grammar now names the units that an SDK must not count. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
NIPOST's autocomplete API returns 37 state codes. For five states they differ from ISO 3166-2:NG. Borno is BR, Gombe GM, Kogi KG, Sokoto SK and Yobe YB. With the ISO codes, parse rejected every real postcode of these states. Each state now also keeps its ISO code in the iso field. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
Reviews of the TypeScript core found wrong implementations that passed every vector. Partial codes could skip the state or segment checks, state names could trim or use Unicode upper case, normalize could upper case before NFKC, and legacy checks could count graphemes or count after NFKC. Each of these now fails at least one vector. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The glossary gains input limit, code point, suggestion and normalise, and it says where the word prefix is allowed. typescript.md now matches the ESLint config: readonly arrays, the extra rules, the Node versions and one spelling for options types. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
NIPOST uses BR, GM, KG, SK and YB where ISO uses BO, GO, KO, SO and YO. New cases show that stateName and parse treat the five ISO codes as unknown, so an SDK cannot keep both lists. The grammar now says that stateName folds the ASCII letters a to z only. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The issue, contributing and security templates named two NIPOST email addresses that NIPOST's developer docs do not list. They now send readers to NIPOST's official website for its contact details. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The five ISO cases were typed by hand in three places, and only BO had a parse case. The builder and the test now read them from states.json, and each ISO code has a parse case. The source text no longer names vehicle plate codes, which nobody verified. The grammar, glossary and TypeScript standard now say what the cases and the config enforce. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
PHP strings hold bytes, and PHP has no NFKC without an extension. The grammar now says what parse, isLegacy and normalize do with text that is not well-formed Unicode, and when an SDK may apply NFKC through a table. A test keeps every normalize vector valid for such an SDK. GIT-1 now names the scopes that commitlint allows. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The new paragraph sat between the lead-in of the parse checks and their list, and it named error codes before the list defined them. It now follows the list. GIT-1 also says that the spec repo, which holds no package, uses no scope. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
NFKC changes with the Unicode version, so a table SDK and a full NFKC on another version could disagree. The grammar now names Unicode 17.0 as the baseline. It also says how broken bytes and lone surrogates meet the input limit, so a Go and a PHP SDK give the same codes. A premise test and a new vector guard the separators. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
No public NIPOST page says that NIPOST uses districts for postal sorting. The pages on postcode.gov.ng describe a district only as a district within the LGA. The glossary now states only what those pages state. The Spec entry named the fixtures and the OpenAPI file as contents of this repo, but the repo holds neither yet. The entry now lists what the repo holds today. It says that the API fixtures and the completed OpenAPI file come later. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The vectors are the contract for every SDK, but nothing in this repo tied them, or the data files, to the grammar. The builder holds its expected results as typed literals, and no test read the segment rules or the suggestion tables. A wrong suggestion in the builder, or a new LGA minimum or legacy pattern in data/, passed make check. scripts/reference.py now reads grammar.md with every value taken from data/, and a test runs each case of each vector file through it. A new vector file fails until the reference has a reader for its function. A second test changes each data value in turn and requires the reference to notice, so a value cannot hide inside the reference. grammar.md repeats some values of data/format.json so that a reader needs no other file. Tests now compare the segment table, the lengths, the input limit, the legacy rule and the fixes with the data. The grammar also says that it wins when a vector disagrees with it. The legacy pattern gets negative cases. The --check mode of the builder gets tests for a changed file, a stray file and a missing file. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
Four wrong readings of the grammar passed every vector. A letter check that looks only at the last character of the area accepted EK01A031K01. An NFKC table without U+208B, U+FE31, U+FE32 and U+FE58 kept four characters that NFKC turns into a separator that is not ASCII. A table without its 4-byte entries passed too, because no normalize case held a character above U+FFFF. Each of these now has a vector. A test computes the characters of the second kind over all code points, so a new one fails the check on a Python that knows it. Some languages can test the sign bit of a negative zero. An SDK that does so gives lga for an accuracy of -0.0, but a plain comparison gives unit. The grammar now says that -0 counts as 0, and a vector pins it. A test checks that the vector file keeps the sign of the zero. The ids after each new case move up by one, as the vector format allows. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The standards cited the spec's interface section, but the spec had none. The fields of a parse result appeared only in the vectors README. grammar.md now has an Interface section. It names each public function and constant, the fields of a parse result and of a parse error, and the rule that each segment after the precision is null. The standards and the PHP and TypeScript files point to it. The glossary gains the two terms that the section uses, parse result and parse error. Tests check that the section names each function and field that the vectors use. The grammar text also had faults. It said that the vectors test every rule, but no shared vector can test the Unicode version or text that is not well-formed. Each SDK tests these rules itself, and the text now says so. Its Unicode example, U+1CCDA, is already in Unicode 16.0, so it showed no 17.0 difference. U+A7F1 does. Unicode also freezes the NFKC form of an assigned character, so only an added character can differ. A test now rejects a vector character that the running Python leaves unassigned, so the oldest CI Python rejects a character that is newer than Unicode 14.0. The text also lacked the rule for a value in truncate that is not a precision, and a pointer from the count of code points to the rule for broken text. A test now ties the version in the text to the VERSION file. The standards pointed to terms that the spec never listed, and to two documents that the repo does not hold. They now give NIPOST's terms and acceptable use policy with their addresses. They say that the status map and the WordPress floors are not defined yet, and when they will be. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
Most rules of check-tells had no test. A run of one-line changes to the script showed that 24 of 33 hand-written changes passed the tests. The tests now pin each emoji range, limit, name, folder, extension, debug call, skippable rule and marker. A run of 328 one-line changes leaves no survivor. The standards say that check-tells rejects emojis everywhere, and that it checks line length, debug calls and to-do comments in code and config files. The tool did less. It gave no check to phpunit.xml.dist, phpstan.neon and infection.json5, and none to style and markup files such as .css and .html. It gave no emoji check to files such as .txt and .lock. It accepted a to-do in lower case and a fix-me without an issue link. All of these now get the same checks as the files next to them. A code or config file that was not UTF-8 passed without a word, and a bad root or path ended in a traceback. check-tells now reports such a file as a TELL-14 break. It answers a bad root, a missing path or a path outside the root with one line and exit code 2. A variable that held a path object now says what it holds. The lockfile list no longer repeats the names that the .lock suffix already covers. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The repo squash-merges. The title of the pull request becomes the subject on main, and its body becomes the body. A message on a branch never reaches main, and CI checked no message at all. A new workflow now builds the squash message from the title, " (#N)" and the body, and runs check-tells on it. The message needs a Conventional Commits type with no scope, a Signed-off-by line, no text left over from the pull request template, and the ASCII and 72-character rules that already existed. A second job needs a Signed-off-by line in each commit of the pull request. No other message rule applies to a commit, because only the squash message reaches main. The jobs run again when someone edits the title or the body, because such an edit pushes no commit. They leave out the HTML comments of the body, which no reader sees and Renovate adds. The standards mark GIT-1, GIT-2, SEC-6 and DEP-5 as enforced by a tool. GIT-1 and GIT-2 now have the jobs above. zizmor, pinned, now runs in make check. It finds an action that is not pinned to a commit, and text of a pull request that reaches a script. renovate.json opens update pull requests each week, pins the digests of actions and moves the tool versions of the Makefile. Its commits and its pull request bodies pass the new jobs. Renovate runs only after the owner installs its GitHub app. A new Ruff release can add rules and turn CI red on a change that has nothing to do with it, and ubuntu-latest changes image on a date that nobody chooses. Each tool in the Makefile now has an exact version, each runner has a version, each job has a time limit, and the matrix no longer stops at the first failure. make lint and make types now check scripts/check-tells, which has no .py suffix and was skipped. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
Five rules, one review item and a table named tools that only the maintainers use, so an outside contributor could not meet them. Each rule now states the property that it needs. DOC-4 gives the writing rules. API-12, DOC-6, UI-5, NPG-8 and the review checklist give theirs. The table is gone. LIC-1 now says what the repos do. A file whose format has comments starts with the two SPDX lines, and REUSE.toml covers Markdown, JSON and YAML files. The text no longer quotes the licence tag, so it needs no ignore markers. DOC-5 allows the changelog format of the release tool, and it names Changesets. The definition of a message catalogue moves next to the kinds of file, and the two rules that use the term point to it. The check-tells section and the language files use the word code for the files that hold code. The pull request body is now the body of the squash commit on main, so the pull request template holds only that text: a paragraph of prose, a Closes line and a Signed-off-by line. The check refuses the placeholder text, and a test ties the template to the check. The checklist moves into CONTRIBUTING.md as plain sentences. CONTRIBUTING.md and AGENTS.md told a contributor to add a remediation commit after a DCO check. No DCO app runs, and the new job needs a sign-off in each commit, so a remediation commit does not help. Both files now say to run git rebase --signoff, and then to push to the contributor's own branch with --force-with-lease. CONTRIBUTING.md no longer points to a code of conduct that no repo holds. AGENTS.md says that spec/ holds the grammar with its Interface section. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The README lists what each folder holds. It now says that grammar.md ends with the public interface, and that scripts/ holds the plain reading of the grammar that checks each vector. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
CS-3 says that a file stays under about 300 lines. After the last fixes, check_tells.py had 707 lines and its test file had 1679 lines. The code is public, so it must read like a well-made tool and not like one long script. check-tells is now the package scripts/check_tells/. Each module has one job. The modules hold the result of a check, the lines and characters of a text, the kind of a file, the debug calls of TELL-13, the checks of one file, the checks of a message, the reading of files and commits, and the command line. Each table of rules sits in the module that reads it. The tests follow the same split. The sample input and the git helpers that several test modules need have modules of their own. Each new file has fewer than 300 lines. The tool does the same work as before. The command has the same options, exit codes and messages. The wrapper scripts/check-tells keeps its import, and only its docstring changes. The package still exports the names that the tests use. Each moved class and function keeps its syntax tree. Six helpers are now public, because they cross a module border. The number of tests is the same. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The squash check blanked each HTML comment of the pull request body before it checked the body. GitHub keeps the raw body in the commit message on main, so the text of a comment reached main with no check. An emoji in a comment passed. The check now reads each comment as text, like the rest of the body. It leaves out only the comment that Renovate adds at the very end of its body, which has the form <!--renovate-debug:BASE64-->. A Renovate pull request still passes. The GIT-1 text of the standards describes the new rule. The body of a Renovate pull request holds a literal Signed-off-by line. It named a different identity than the commits do. The hosted app commits as renovate[bot], so renovate.json now writes that identity in the body. The commits keep their sign-off through commitTrailers. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
A digit check without its end anchor reads the first digit of a segment and ignores the rest. Such a check accepted JI-9X-CHK-LS-18, and it passed every vector. The only earlier cases with a letter after a digit have a unit that starts with 0, which fails the minimum on its own. Two vectors close the gap. JI9XCHKLS18 has an LGA of 9X, and JI99CHKLS1X has a unit of 1X. Both give bad_segment. The letter X has no fix, so neither code gets a suggestion. A copy of the reference reading with the weak check passes the 306 earlier vectors and fails exactly these two. A test checks that the file keeps a case of each kind. The ids from parse-segments-015 on move up by two, as the vector format allows. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
Renovate can add its own notes to the notes section of the body, after the sign-off line, and those notes can hold characters that the squash message check rejects. The sign-off now sits in the header, and the body template ends with it, as in the js repo. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The js repo bans require in package code, because an ES module has no require at run time and the Node types make tsc accept it. The tool table now lists it with the other banned globals. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
MobileMage
changed the title
The squash check leaves out the comment that Renovate adds to the end of its pull request body. The pattern for that comment ended with \s*$. In Python, \s matches Unicode white space. A body that ended with the comment and then U+3000, U+2028, U+00A0, U+0085 or U+2003 lost that character together with the comment. TELL-14 never saw it, and the character reached main. Any author can write the comment. The pattern now allows only a space, a tab, a carriage return or a line feed after the comment. Then the text must end. TELL-14 reports any other character. A test covers each of the five characters. A second test covers the ASCII white space that still passes. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
The standards listed the placeholder paragraph and the empty "Closes #" line of the pull request template under GIT-2. The tool reports both as TELL-18, which asks for the reason in the body, in prose. The text now sits under TELL-18. The standards also said that data files can use any script. The tool holds a JSON data file, such as data/states.json, to ASCII, because a .json file is a config file. The text now says so, and it names .csv as a data file that can use any script. The contributing guide gave only titles with a scope. The spec repo has no packages, and its squash check rejects a scope, so a contributor who copied the example got a GIT-1 failure. The guide now shows the title that the spec repo uses. The README of the vectors said that a case that the grammar contradicts fails the check. A case fails when the plain reading in scripts/reference.py disagrees with it, and the README now says that. The grammar said that the vectors test every rule, with two exceptions. At least three more rules have no shared vector: normalize accepts input of any length, truncate fails for a value that is not a precision, and each SDK exposes SPEC_VERSION. The grammar now lists all five and says that each SDK tests them itself. New tests tie each of these statements to the tool, so that the text and the tool cannot drift apart again without a failing test. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
Two mistakes passed every shared vector. The grammar says that normalize has no input limit, but no normalize case had more than 64 code points. A reading whose normalize refused long text, or cut it at the limit, passed all 308 vectors. The new case normalize-075 puts 64 spaces on each side of a code, so the code starts after the limit. Both mistakes fail it. A contains that looks for the prefix anywhere in the code, and not at its start, passed every vector too. The new case contains-009 has the prefix FC and the code EK-01-FC0-FK-01. The district FC0 starts with FC, so such a contains says true, and the right answer is false. The grammar named the first rule as one that no vector tests. It now names only the rules that still have no shared vector. Tests plant each mistake in a copy of the reference reading and require that a vector catches it. Both cases come last in their files, so no earlier id moves. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
A contains that skips the LGA or the area of the prefix passed every vector. Two cases now give a prefix that differs from the code only in that segment, so such a check finds the code inside it and fails. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
Ten lower-case letters, such as z and q, were in no vector input, so a case table that lacked one of them passed every vector. Za is the code of Zamfara, so such a table would reject real input. Signed-off-by: MobileMage <aidelojejoshua@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gatepost is a set of unofficial SDKs for Nigeria's digital postcodes.
This repo is the spec that each SDK follows. grammar.md says how an SDK
reads and writes a postcode, and it ends with the public interface.
data/ holds the values: NIPOST's 37 state codes, the segment rules, the
separators, the input limit and the GPS precision limits. vectors/
holds the cases that every SDK must pass. A plain reading of the
grammar in scripts/reference.py runs each vector against the data, so a
vector or a data value that disagrees with the grammar fails the check.
The coding standards and the language files set the rules for every
repo. check-tells finds the habits that make code look machine-written,
such as long lines, narrating comments and emojis. CI runs the tests,
the vector builder, check-tells, Ruff, mypy, REUSE, zizmor and gitleaks.
It also checks this message, which the squash merge puts on main, and
the sign-off of each commit.
This change is larger than the 400 lines that GIT-5 suggests, because
it starts the repo. Most of it is vectors, tests and standards text.
scripts/build_vectors.py passes the 300 lines that CS-3 suggests,
because most of its lines are rows of vector cases. Its test file
passes it too, with one test for each property of the vectors.
scripts/reference.py has 328 lines, because it reads the whole grammar
and has one small reader for each function that the vectors test.
Signed-off-by: MobileMage aidelojejoshua@gmail.com