The Heuristics Behind the Security Risk Scanner

ArchSetu's security scanner is deliberately not a real SAST tool. It doesn't build an AST, it doesn't track data flow from a request parameter to a dangerous sink, and it makes zero network calls - no CVE database, nothing. It runs regex patterns against raw line text across five categories: hardcoded secrets, unsafe dynamic execution, string-built SQL, insecure transport, and weak cryptography. That's a real, stated tradeoff, not a limitation we're hiding: zero network calls and zero added dependencies, in exchange for false positives being expected and accepted as the cost of a signal a human still has to look at.

The interesting part isn't the regex list itself - it's what we learned running it against real, large open-source codebases, and the two genuine bugs that surfaced as a result.

The five categories, briefly

• Hardcoded secrets - API key patterns (AWS, GitHub, Google, Stripe, Slack, ...), JWTs, database connection strings with embedded credentials, and a generic "credential-shaped assignment" rule for anything else.
• Dynamic code execution - eval(), exec(), unsafe deserialization (pickle, Marshal.load), and language-specific equivalents across 14 languages.
• String-built SQL - queries assembled via concatenation or interpolation instead of parameterized statements.
• Insecure transport - disabled certificate verification (verify=False, rejectUnauthorized: false, InsecureSkipVerify: true).
• Weak cryptography - MD5/SHA-1 for anything security-sensitive, DES/RC4, and Math.random() used to build a token.

Bug one: comments and test fixtures, found on authelia/authelia

Authelia is itself a security-focused authentication project - a genuinely good, adversarial test case. Running the scanner against it surfaced 42 findings. Manually checking every one found that roughly 40 of them were not real:

• About 16 were literal YAML comments inside config.template.yml - a config template's entire purpose is showing example values, many deliberately left commented out. The scanner was reading commented-out example secrets as if they were live ones.
• About 24 more were in test fixture files - Go test suites with variables named things like x509PrivateKeyRSAExpired, containing intentionally-fake certificates used to verify the project's own parsing logic handles expired/malformed input correctly. Fake, on-purpose, embedded in a test file - not a leak.

The fix was two skip conditions: a comment-line check (covering #, //, --, and a few others across supported languages) applied before every rule, and a full-file skip for anything matching common test/fixture directory conventions (/test/, /tests/, /testdata/, /__tests__/, and several kernel/systems-specific ones like /selftests/ found separately by testing against torvalds/linux).

Bug two: compound directory names, found on go-gitea/gitea

A second pass, this time against Gitea, surfaced 29 findings. Gitea's own migration package lives at modelmigration/ - a single compound word, not the conventional /migrations/ or /migration/ directory the scanner already knew to skip (migration scripts build DDL by interpolating table/column names, which is simply how migrations are written, not evidence of unparameterized user input). Because there's no literal / directly before "migration" in that path, the existing exact-segment check never matched it.

The same pattern repeated for Gitea's test-helper package, models/unittest/ - a directory name the existing /test/ and /tests/ checks don't cover either, since unittest is a distinct word.

Both fixes were narrow and specific: the migration check now matches any path segment containing "migration" as a substring rather than requiring an exact match, and /unittest/ was added alongside the existing test-directory patterns. After both fixes, Gitea's 29 findings dropped to 8 - all genuinely ambiguous internal database-bookkeeping code we couldn't confidently rule out without more context, not resolved by pattern-matching a directory name.

The honest limit

Even after both fixes, this scanner cannot tell you whether a flagged line is reachable by untrusted input - that requires real data-flow analysis, which a regex pass structurally cannot do. Every finding is "this pattern is a well-established anti-pattern, worth a human look," never "this is a confirmed vulnerability." That's not a hedge - it's the actual, stated design contract, and it's why every finding on a public report only shows severity counts to anyone who isn't the verified repository owner.