Architecture for a fast, low-noise N+1 Query Linter #1715
kiarash8112
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hey everyone,
I've been looking into building a custom static analysis linter to detect N+1 database queries in our Go codebase.
The standard naive approach—finding a
forloop and recursively walking the AST/Call Graph for every function call inside it to see if it hits adb.Query—has two fatal flaws:I'd like to propose a two-stage architectural approach to solve both the performance and accuracy problems
Stage 1: The Fast Filter (Pre-computed Call Graph)
Goal: Solve the O(N^2) speed problem by doing the heavy lifting once.
Instead of tracing top-down from every loop, we work bottom-up from our database sinks.
*sql.DB,*sql.Tx, etc. (our sinks).TransitivelyCallsDB = map[funcName]bool.Now, when our linter visits a loop, checking if
myHelperFunc()is dangerous is an instantO(1)map lookup. If it's not in the set, we skip it entirely.Stage 2: The Accuracy Filter (Dataflow/Taint Tracking)
Goal: Eliminate false positives by proving the loop actually drives the query.
If a function call inside a loop passes Stage 1, we still need to prove it's a true N+1.
We apply basic taint analysis:
iteminfor _, item := range items).If the database query arguments contain tainted data (e.g.,
db.Query("... id = ?", item.ID)), it is a high-confidence N+1. If the query arguments are clean/static (e.g.,db.Query("SELECT version()")), we drop the warning as a false positive.Questions for the community:
Would love to hear your thoughts on this architecture before starting the implementation.
Beta Was this translation helpful? Give feedback.
All reactions