Skip to main content

June 2021

· 6 min read

The following updates were made to Semgrep in June 2021.

Version 0.57.0​

Additions​

  • New options: field in a YAML rule to enable/disable certain features (for example constant propagation) (For the list of available features you can enable/disable see https://github.com/semgrep/semgrep/blob/develop/interfaces/Rule_options.atd).
  • Capture groups in pattern-regex: in $1, $2, etc. (#3356)
  • Support metavariables inside atoms (e.g., foo(:$ATOM))
  • Support metavariables and ellipsis inside regexp literals (for example foo(/.../))
  • Associative-commutative matching for bitwise OR, AND, and XOR operations
  • Add support for $...MVAR in generic patterns
  • Add support for $...MVAR in generic patterns
  • metavariable-pattern: Add support for nested Spacegrep/regex/Comby patterns
  • C#: support ellipsis in method parameters (#3289)

Fixes​

  • C#: parse __makeref, __reftype, __refvalue (#3364)
  • Java: parsing of dots inside function annotations with brackets (#3389)
  • Do not pretend that short-circuit Boolean AND and OR operators are commutative (#3399)
  • metavariable-pattern: Fix crash when nesting a non-generic pattern within a generic rule
  • metavariable-pattern: Fix parse info when matching content of a metavariable under a different language
  • generic mode on Markdown files with very long lines will now work (#2987)

Changes​

  • generic mode: files that don't look like nicely-indented programs are no longer ignored, which may cause accidental slowdowns in setups where excessively large files are not excluded explicitly (#3418)
  • metavariable-comparison: Fix crash when comparing integers and floats
  • Do not filter findings with the same range but different metavariable bindings (#3310)
  • Set parsing_state.have_timeout when a timeout occurs (#3438)
  • Set a timeout of 10s per file (#3434)
  • Improvements to contributing documentation (#3353)
  • Memoize getting ranges to speed up rules with large ranges
  • When and-ed with other patterns, pattern: $X will not be evaluated on its own, but will look at the context and find $X within the metavariables bound, which should be significantly faster

Version 0.56.0​

Additions​

  • Associative-commutative matching for Boolean AND and OR operations (#3198)
  • Support metavariables inside strings (e.g., foo("$VAR"))
  • Support metavariables inside atoms (e.g., foo(:$ATOM))
  • metavariable-pattern: allow matching the content of a metavariable under a different language

Fixes​

  • C#: Parse attributes for local functions (#3348)
  • Go: Recognize other common package naming conventions (#2424)

Changes​

  • Upgraded TypeScript parser (#3102)

Version 0.55.1​

Additions​

  • Added new metavariable-pattern operator (available only via --optimizations), thanks to Kai Zhong for the feature request (#3257)
  • Add helpUri to SARIF output if rule source metadata is defined

Fixes​

  • C#: Support unsafe block syntax (#3283)
  • Generic mode: fixed wrong line numbers for multi-lines match (#3315)
  • JavaScript: support partial field definitions pattern, like in JSON
  • JSON: handle correctly metavariables as field (#3279)
  • PHP: Support ellipsis in include/require and echo (#3191,#3245)
  • PHP: Prefer expression patterns over statement patterns (#3191)
  • Python: support ellipsis in try-except (#3233)
  • Scala: correctly parse symbol literals and interpolated strings containing double dollars (#3271)
  • Taint mode: Allow statement-patterns when these are represented as statement-expressions in the Generic AST (#3191)
  • Dataflow: Analyze foreach body even if we do not handle the pattern yet (#3155)
  • Correctly handle ellipsis inside function types (#3119)
  • Fall back to no optimizations when using unsupported features: pattern-where-python, taint rules, and --debugging-json (#3265)
  • Handle regexp parse errors gracefully when using optimizations (#3266)
  • Support equivalences when using optimizations (#3259)

Changes​

  • Run rules in semgrep-core (rather than patterns) by default (these are the optimizations described above)

Version 0.54.0​

This version includes release notes for Semgrep version 0.53.0 as well.

Additions​

  • Alpha support for Scala
  • Metrics collection of project_hash in cases where git is not available
  • Taint mode now also analyzes top-level statements
  • Per rule parse times and per rule-file parse and match times added to opt-in metrics
  • $...MVAR can now match a list of statements (not just a list of arguments) (#3170)

Fixes​

  • JavaScript parsing: Support decorators on properties
  • JavaScript parsing: Allow default export for any declaration
  • Metavariables in messages are filled in when using --optimizations all
  • Respect --timeout-threshold option in --optimizations all mode
  • Python: class variables are matched in any order (#3212)
  • Running with --strict will now return results if there are nosem mismatches. Semgrep will report a nonzero exit code if --strict is set and there are nosem mismatches (#3099)
  • PHP: parsing correctly ... and metavariables in parameters
  • PHP: parsing correctly functions with a single statement in their body
  • Evaluate interpolated strings during constant propagation (#3127)
  • Semgrep will report an InvalidRuleSchemaError for dictionaries with duplicate key names (#3084)
  • Basic type inference also for implicit variable declarations (Python, Ruby, PHP, and JavaScript)
  • JavaScript/TypeScript: differentiating tagged template literals in the AST (#3187)
  • Ruby: storing parenthesis in function calls in the AST (#3178)

Changes​

  • Moved some debug logging to verbose logging
  • $...ARGS can now match an empty list of arguments, just like ... (#3177)
  • JSON and SARIF outputs sort keys for predictable results