[{"data":1,"prerenderedAt":771},["ShallowReactive",2],{"/en-us/blog/a-benchmarking-framework-for-sast/":3,"navigation-en-us":41,"banner-en-us":470,"footer-en-us":487,"Michael Henriksen-Martynas Krupskis-Mark Art-Dinesh Bolkensteyn-Isaac Dawson-Julian Thome":697,"next-steps-en-us":756},{"_path":4,"_dir":5,"_draft":6,"_partial":6,"_locale":7,"seo":8,"content":16,"config":31,"_id":34,"_type":35,"title":36,"_source":37,"_file":38,"_stem":39,"_extension":40},"/en-us/blog/a-benchmarking-framework-for-sast","blog",false,"",{"title":9,"description":10,"ogTitle":9,"ogDescription":10,"noIndex":6,"ogImage":11,"ogUrl":12,"ogSiteName":13,"ogType":14,"canonicalUrls":12,"schema":15},"A Google Summer of Code project: creating a benchmarking framework for SAST","Our 2022 Google Summer of Code project helped to create a benchmarking framework for SAST.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1749677267/Blog/Hero%20Images/benchmarking.png","https://about.gitlab.com/blog/a-benchmarking-framework-for-sast","https://about.gitlab.com","article","\n                        {\n        \"@context\": \"https://schema.org\",\n        \"@type\": \"Article\",\n        \"headline\": \"A Google Summer of Code project: creating a benchmarking framework for SAST\",\n        \"author\": [{\"@type\":\"Person\",\"name\":\"Michael Henriksen\"},{\"@type\":\"Person\",\"name\":\"Martynas Krupskis\"},{\"@type\":\"Person\",\"name\":\"Mark Art\"},{\"@type\":\"Person\",\"name\":\"Dinesh Bolkensteyn\"},{\"@type\":\"Person\",\"name\":\"Isaac Dawson\"},{\"@type\":\"Person\",\"name\":\"Julian Thome\"}],\n        \"datePublished\": \"2022-09-27\",\n      }",{"title":9,"description":10,"authors":17,"heroImage":11,"date":24,"body":25,"category":26,"tags":27},[18,19,20,21,22,23],"Michael Henriksen","Martynas Krupskis","Mark Art","Dinesh Bolkensteyn","Isaac Dawson","Julian Thome","2022-09-27","In summer 2022, the [Vulnerability Research team at\nGitLab](/handbook/engineering/development/sec/secure/vulnerability-research/) \n\nlaunched the [Google Summer of Code\n(GSoC)](https://summerofcode.withgoogle.com/) project: \n\n[A benchmarking framework for\nSAST](https://gitlab.com/gitlab-com/marketing/community-relations/contributor-program/gsoc-2022/-/issues/1).\n\n\nThe goal of the project was to create a benchmarking framework, which would\nassess the\n\nimpact and quality of a security analyzer or configuration change before it\nreaches the production environment.\n\n\n## Preliminaries \n\n\n### GitLab SAST\n\n\nAs a complete DevOps Platform, GitLab has a variety of integrated [static\nanalysis (SAST) tools](/direction/secure/static-analysis/sast/) \n\nfor different languages and frameworks. These tools help developers find\n\nvulnerabilities as early as possible in the software development lifecycle.\n\nThese tools are constantly being updated, either by upgrading the underlying\n\nsecurity analyzers or by applying configuration changes.\n\n\nSince all the integrated SAST tools are very different in terms of\n\nimplementation, and depend on different tech stacks, they are all\n\nwrapped in Docker images. The wrappers translate tool-native vulnerability\n\nreports to a [generic, common report\nformat](https://docs.gitlab.com/ee/user/application_security/sast/#reports-json-format)\nwhich is\n\nmade available by means of the `gl-sast-report.json` artifact. This generic\n\nreport is GitLab's common interface between analyzers and the GitLab Rails\n\nbackend.\n\n\nBenchmarking is important to assess the efficacy of analyzers and helps to\nmake\n\ndata-driven decisions. For example, benchmarking is useful for QA testing\n\n(spotting regressions), for data-driven decision making, and for research by\n\nassessing the progression of the GitLab security feature performance over\ntime.\n\n\n### Google Summer Of Code (GSoC)\n\n\n[Google Summer of Code (GSoC)](https://summerofcode.withgoogle.com/) \n\nis a 10-week program that enlists contributors to work on open source\nprojects\n\nin collaboration with open source organizations. For GSoC 2022, GitLab\noffered\n\nfour projects to GSoC contributors. The contributors completed each of the\n\nprojects with the guidance from GitLab team members who mentored them and\n\nprovided regular feedback and assistance when needed.\n\n\n### Terms & Notation\n\n\nIn this blog post, we use the terms/acronyms below to classify findings\nreported by security analyzers.\n\n\n| Acronym   | Meaning        |\nDescription                                                        |\n\n|-------|----------------|--------------------------------------------------------------------|\n\n| _TP_  | True Positive  | Analyzer correctly identifies a\nvulnerability.                     |\n\n| _FP_  | False Positive | Analyzer misidentifies a vulnerability or\nreported a vulnerability where none exist. |\n\n| _TN_  | True Negative  | Analyzer correctly ignores a potential false\npositive.             |\n\n| _FN_  | False Negative | Analyzer does not report a known\nvulnerability.                    |\n\n\nFor the figures in the blog post we use the following notation: processes\nare\n\ndepicted as rounded boxes, whereas artifacts (e.g., files) are depicted as\n\nboxes; arrows denote an input/output (IO) relationship between the connected\nnodes.\n\n\n``` mermaid\n\nflowchart TB;\n\nsubgraph legend[ Legend ]\n   proc(Process);\n   art[Artifact];\n   proc -->|IO relation|art;\nend\n\n``` \n\n\n## Motivation\n\n\nThe authors of the paper [How to Build a\nBenchmark](https://dl.acm.org/doi/10.1145/2668930.2688819) distilled the\ndesirable characteristics of a benchmark below:\n\n> 1. Relevance: How closely the benchmark behavior correlates to behaviors\nthat are of interest to consumers of the results.\n\n> 2. Reproducibility: The ability to consistently produce similar results\nwhen the benchmark is run with the same test configuration.\n\n> 3. Fairness: Allowing different test configurations to compete on their\nmerits without artificial limitations.\n\n> 4. Verifiability: Providing confidence that a benchmark result is\naccurate.\n\n> 5. Usability: Avoiding roadblocks for users to run the benchmark in their\ntest environments.\n\n\nThere currently is no standard nor de facto language-agnostic SAST benchmark\n\nsatisfying all the criteria mentioned above. Many benchmark suites focus on\n\nspecific languages, are shipped with incomplete or missing ground-truths, or\n\nare based on outdated technologies and/or frameworks. A ground-truth or\n\nbaseline is the set of findings a SAST tool is expected to detect.\n\n\nThe main objective of the GSoC project was to close this gap and start to\n\ncreate a benchmarking framework that addresses all the desirable\ncharateristics\n\nmentioned above in the following manner:\n\n\n1. Relevance: Include realistic applications (in terms of size, framework\nusage\n   and customer demand).\n2. Reproducibility: Automate the whole benchmarking process in CI.\n\n3. Fairness: Make it easy to integrate new SAST tools by just tweaking the\nCI\n   configuration and use the [GitLab security report schema](https://docs.gitlab.com/ee/user/application_security/sast/#reports-json-format) as a common standard.\n4. Verifiability: Assemble baseline that includes all the relevant\n   vulnerabilities and make it publicly available. The baseline is the north star\n   that defines what vulnerabilities are actually included in a test application. \n5. Usability: Benchmark users can just integrate the benchmark as a\ndownstream\n   pipeline to their CI configuration.\n\n## A benchmarking framework for SAST\n\n\nThe benchmarking framework compares the efficacy of an analyzer against a\nknown\n\nbaseline. This is very useful for monitoring the efficacy of the analyzer\nthat\n\nparticipates in the benchmarking. The baseline is the gold standard that\nserves\n\nas a compass to guide analyzer improvements.\n\n\n### Usage\n\n\nFor using the framework, the following requirements have to be met:\n\n1. The analyzer has to be dockerized.\n\n1. The analyzer has to produce a vulnerability report that adheres to the\n   [GitLab security report schema](https://docs.gitlab.com/ee/user/application_security/sast/#reports-json-format)\n   format, which serves as our generic intermediate representation to compare\n   analyzer efficacy. \n1. The baseline expectations have to be provided as \n   [GitLab security report schema](https://docs.gitlab.com/ee/user/application_security/sast/#reports-json-format)\n   so that we can compare the analyzer output against it. \n\nThe framework is designed in such a way that it can be easily integrated\ninto\n\nthe CI configuration of existing GitLab projects by means of a [downstream\npipeline](https://docs.gitlab.com/ee/ci/pipelines/downstream_pipelines.html).\n\nThere are many possible ways in which a downstream pipeline can be\ntriggered:\n\nsource code changes applied to an analyzer, configuration changes\n\napplied to an analyzer, or scheduled pipeline invocation. By using the\npipeline,\n\nwe can run the benchmarking frameworks continuously and instantaneously on\nthe GitLab\n\nprojects that host the source code of the integrated analyzers whenever code\nor\n\nconfiguration changes are applied. \n\n\n### Architecture \n\n\nThe figure below depicts the benchmarking framework when comparing an\nanalyzer\n\nagainst a baseline.\n\n\nWe assume that we have a baseline configuration available; a baseline\nconsists\n\nof an application that is an actual test application that includes\n\nvulnerabilities. These vulnerabilities are documented in an expectation file\n\nthat adheres to the [security report\nschema](https://docs.gitlab.com/ee/user/application_security/sast/#reports-json-format).\n\n\nNote that we use the terms baseline and expectation interchangeably. As\n\nmentioned earlier, the benchmarking framework is essentially a GitLab\npipeline\n\nthat can be triggered downstream. The configured analyzer then takes the\n\nbaseline app as input and generates a `gl-sast-report.json` file. The heart\nof the\n\nbenchmarking framework is the `compare` step, which compares the baseline\n\nagainst the report generated by the analyzer, both of which adhere to the\n\n[security report\nschema](https://docs.gitlab.com/ee/user/application_security/sast/#reports-json-format).\n\n\nThe compare step also computes the _TP_, _FN_ and _FP_ that have been\nreported by the\n\nanalyzer and computes different metrics based on this information. The\ncompare\n\nstep is implemented in the\n\n[evaluator\ntool](https://gitlab.com/gitlab-org/secure/gsoc-sast-benchmark/evaluator). \n\n\n``` mermaid\n\nflowchart LR;\n\nsbx[gl-sast-report.json];\n\nbreport[Report];\n\nconfig[Configuration];\n\n\nconfig --> bf;\n\n\nsubgraph Baseline\n  bcollection[app];\n  baseline[expectation];\nend\n\n\nsubgraph bf [ Benchmarking Framework ]\n   orig(Analyzer);\n   compare(Compare);\n   orig --> sbx;\n   sbx --> compare;\nend\n\n\nbaseline --> compare;\n\ncompare --> breport\n\nbcollection --> orig\n\n```\n\n\nUsing the security report format as a common standard makes the benchmarking\n\nframework very versatile: the baseline could be provided by an automated\n\nprocess, by another analyzer, or manually, which happened to be the case in\nthis\n\nGSoC project.\n\n\n### Scoring\n\n\nThe main functionality of the [evaluator\ntool](https://gitlab.com/gitlab-org/secure/gsoc-sast-benchmark/evaluator)\n\nis to compute the overlap/intersection, and difference between a baseline\nand\n\ngenerated report in order to uncover true positives, false positives, and\nfalse\n\nnegatives. \n\n\nThe relationship between _TP_, _FP_, _FN_, _TN_, baseline, and generated\nreport can be\n\nseen in the table below; it includes three columns `analyzer`, `baseline`\nand\n\n`classification`. The column `analyzer` represents the findings included in\nthe\n\nreport generated by the analyzer; column `baseline` represents the findings\n\nincluded in the baseline; column `classification` denotes the\n\nverdict/classification that the [evaluator\ntool](https://gitlab.com/gitlab-org/secure/gsoc-sast-benchmark/evaluator)\n\nattaches to the analyzer finding when performing the comparison. The `X` and\n\n`-` denote reported and non-reported findings, respectively.\n\n\n| analyzer | baseline | classification |\n\n| -------- | -------  | -------------  |\n\n| -        | -        | _TN_           |\n\n| -        | X        | _FN_           |\n\n| X        | -        | _FP_           |\n\n| X        | X        | _TP_           |\n\n\nThe `classification` column in the table above shows that a _TP_ is a\n\nvulnerability existing in both baseline and generated report; similarly, an\n\n_FP_ is a vulnerability detected by an analyzer without a corresponding\n\nbaseline entry, while an _FN_ is a vulnerability present in the baseline but\n\nnot detected by an analyzer. Note, that _TN_ is practically not relevant for\n\nour use-case since the analyzers we are looking at only report unsafe,\n\nvulnerable cases instead of safe, non-vulnerable cases. \n\n\nAt the moment, the `evaluator` tool computes the metrics below:\n\n- Precision: _P_ = _TP_ /( _TP_ + _FP_ )\n\n- Recall: _R_ = _TP_ / ( _TP_ + _FN_ )\n\n- F-Score: _F_ = 2 * ( _P_ * _R_ ) / ( _P_ + _R_ ) \n\n- Jaccard-Index: _J_ = _TP_ / ( _TP_ + _FP_ + _FN_ )\n\n\nA higher precision indicates that an analyzer is less noisy due to the\nlow(er)\n\nnumber of _FPs_. Hence, a high precision leads to a reduction of auditing\neffort\n\nof irrelevant findings. A high recall represents an analyzer's detection\n\ncapacity. F-Score is a combined measure so that precision and recall can be\n\ncondensed to a single number. The Jaccard-Index is a single value to capture\n\nthe similarity between analyzer and baseline.\n\n\nThe [evaluator\ntool](https://gitlab.com/gitlab-org/secure/gsoc-sast-benchmark/evaluator)\n\nsupports the addition of custom metrics via a simple call-back mechanism;\nthis\n\nenables us to add support more metrics in the future that help us to gain\n\nadditional or new insights with regards to the efficacy of our analyzers.\n\n\n### Framework Properties\n\n\nIn principle, the implemented benchmarking framework is language-agnostic:\nnew\n\nanalyzers and baselines can be plugged-in as long as they adhere to the\n\n[security report\nschema](https://docs.gitlab.com/ee/user/application_security/sast/#reports-json-format). \n\n\nEstablishing baselines is laborious since it requires (cross-)validation, \n\ntrying out attacks on the running baseline application and\n\ncode auditing.\n\n\nFor the GSoC project, we established baselines for the applications below\n\ncovering Java ([Spring](https://spring.io/)) and Python\n\n([Flask](https://flask.palletsprojects.com/)) as they are [ranking high in\nthe most used languages and\nframeworks](https://survey.stackoverflow.co/2022/#technology-most-popular-technologies). \n\nFor a benchmark application to have practical utility, it is important that\nthe\n\napplication itself is based on technology, including programming languages\nand\n\nframeworks, that are used in the industry.\n\n\nFor both of these applications, the baseline/expectations have been\ncollected,\n\nverified and are publicly availabe: \n\n-\n[WebGoat](https://gitlab.com/gitlab-org/secure/gsoc-sast-benchmark/baselines/WebGoat/-/tree/baselines). \n  WebGoat is a deliberately insecure Web application used to teach security vulnerabilities.\n  We chose this as baseline application because it is often used as a benchmark\n  app in the Java world and it is based on [Spring](https://spring.io/) which is\n  one of the most popular frameworks in the Java world. \n-\n[vuln-flask-web-app](https://gitlab.com/gitlab-org/secure/gsoc-sast-benchmark/baselines/vuln-flask-web-app/-/tree/report)\nLike WebGoat, this application is deliberately insecure.\n`vuln-flask-web-app` covers both Python and\n[Flask](https://flask.palletsprojects.com/en/2.2.x/), one of the most\npopular web frameworks in the Python world.\n\n\n## Conclusion\n\n\nThis GSoC project was a first step towards building a FOSS benchmarking\n\nframework that helps the community to test their own tools and to build up a\n\nrelevant suite of baselines covering various languages and frameworks. With\nthe\n\nhelp of the community, we will continue adding more baselines to the\n\nbenchmarking framework in the future to cover more languages and frameworks.\n\n\nIf you found the project interesting, you might want to check out the\nfollowing repositories:\n\n\n-\n[evaluator](https://gitlab.com/gitlab-org/secure/gsoc-sast-benchmark/evaluator)\n\n- [WebGoat\nbaseline](https://gitlab.com/gitlab-org/secure/gsoc-sast-benchmark/baselines/WebGoat/-/tree/baselines)\n\n- [Vulnerable Flask Web App\nbaseline](https://gitlab.com/gitlab-org/secure/gsoc-sast-benchmark/baselines/vuln-flask-web-app/-/tree/report)\n\n- [Example of downstream pipeline triggering\nevaluator](https://gitlab.com/gitlab-org/secure/gsoc-sast-benchmark/evaluator-downstream)\n\n\nCover image by [Maxim Hopman](https://unsplash.com/@nampoh) on\n[Unsplash](https://unsplash.com/photos/fiXLQXAhCfk)\n\n{: .note}\n","open-source",[28,29,30],"collaboration","google","open source",{"slug":32,"featured":6,"template":33},"a-benchmarking-framework-for-sast","BlogPost","content:en-us:blog:a-benchmarking-framework-for-sast.yml","yaml","A Benchmarking Framework For Sast","content","en-us/blog/a-benchmarking-framework-for-sast.yml","en-us/blog/a-benchmarking-framework-for-sast","yml",{"_path":42,"_dir":43,"_draft":6,"_partial":6,"_locale":7,"data":44,"_id":466,"_type":35,"title":467,"_source":37,"_file":468,"_stem":469,"_extension":40},"/shared/en-us/main-navigation","en-us",{"logo":45,"freeTrial":50,"sales":55,"login":60,"items":65,"search":397,"minimal":428,"duo":447,"pricingDeployment":456},{"config":46},{"href":47,"dataGaName":48,"dataGaLocation":49},"/","gitlab logo","header",{"text":51,"config":52},"Get free trial",{"href":53,"dataGaName":54,"dataGaLocation":49},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com&glm_content=default-saas-trial/","free trial",{"text":56,"config":57},"Talk to sales",{"href":58,"dataGaName":59,"dataGaLocation":49},"/sales/","sales",{"text":61,"config":62},"Sign in",{"href":63,"dataGaName":64,"dataGaLocation":49},"https://gitlab.com/users/sign_in/","sign in",[66,110,208,213,318,378],{"text":67,"config":68,"cards":70,"footer":93},"Platform",{"dataNavLevelOne":69},"platform",[71,77,85],{"title":67,"description":72,"link":73},"The most comprehensive AI-powered DevSecOps Platform",{"text":74,"config":75},"Explore our Platform",{"href":76,"dataGaName":69,"dataGaLocation":49},"/platform/",{"title":78,"description":79,"link":80},"GitLab Duo (AI)","Build software faster with AI at every stage of development",{"text":81,"config":82},"Meet GitLab Duo",{"href":83,"dataGaName":84,"dataGaLocation":49},"/gitlab-duo/","gitlab duo ai",{"title":86,"description":87,"link":88},"Why GitLab","10 reasons why Enterprises choose GitLab",{"text":89,"config":90},"Learn more",{"href":91,"dataGaName":92,"dataGaLocation":49},"/why-gitlab/","why gitlab",{"title":94,"items":95},"Get started with",[96,101,106],{"text":97,"config":98},"Platform Engineering",{"href":99,"dataGaName":100,"dataGaLocation":49},"/solutions/platform-engineering/","platform engineering",{"text":102,"config":103},"Developer Experience",{"href":104,"dataGaName":105,"dataGaLocation":49},"/developer-experience/","Developer experience",{"text":107,"config":108},"MLOps",{"href":109,"dataGaName":107,"dataGaLocation":49},"/topics/devops/the-role-of-ai-in-devops/",{"text":111,"left":112,"config":113,"link":115,"lists":119,"footer":190},"Product",true,{"dataNavLevelOne":114},"solutions",{"text":116,"config":117},"View all Solutions",{"href":118,"dataGaName":114,"dataGaLocation":49},"/solutions/",[120,145,169],{"title":121,"description":122,"link":123,"items":128},"Automation","CI/CD and automation to accelerate deployment",{"config":124},{"icon":125,"href":126,"dataGaName":127,"dataGaLocation":49},"AutomatedCodeAlt","/solutions/delivery-automation/","automated software delivery",[129,133,137,141],{"text":130,"config":131},"CI/CD",{"href":132,"dataGaLocation":49,"dataGaName":130},"/solutions/continuous-integration/",{"text":134,"config":135},"AI-Assisted Development",{"href":83,"dataGaLocation":49,"dataGaName":136},"AI assisted development",{"text":138,"config":139},"Source Code Management",{"href":140,"dataGaLocation":49,"dataGaName":138},"/solutions/source-code-management/",{"text":142,"config":143},"Automated Software Delivery",{"href":126,"dataGaLocation":49,"dataGaName":144},"Automated software delivery",{"title":146,"description":147,"link":148,"items":153},"Security","Deliver code faster without compromising security",{"config":149},{"href":150,"dataGaName":151,"dataGaLocation":49,"icon":152},"/solutions/security-compliance/","security and compliance","ShieldCheckLight",[154,159,164],{"text":155,"config":156},"Application Security Testing",{"href":157,"dataGaName":158,"dataGaLocation":49},"/solutions/application-security-testing/","Application security testing",{"text":160,"config":161},"Software Supply Chain Security",{"href":162,"dataGaLocation":49,"dataGaName":163},"/solutions/supply-chain/","Software supply chain security",{"text":165,"config":166},"Software Compliance",{"href":167,"dataGaName":168,"dataGaLocation":49},"/solutions/software-compliance/","software compliance",{"title":170,"link":171,"items":176},"Measurement",{"config":172},{"icon":173,"href":174,"dataGaName":175,"dataGaLocation":49},"DigitalTransformation","/solutions/visibility-measurement/","visibility and measurement",[177,181,185],{"text":178,"config":179},"Visibility & Measurement",{"href":174,"dataGaLocation":49,"dataGaName":180},"Visibility and Measurement",{"text":182,"config":183},"Value Stream Management",{"href":184,"dataGaLocation":49,"dataGaName":182},"/solutions/value-stream-management/",{"text":186,"config":187},"Analytics & Insights",{"href":188,"dataGaLocation":49,"dataGaName":189},"/solutions/analytics-and-insights/","Analytics and insights",{"title":191,"items":192},"GitLab for",[193,198,203],{"text":194,"config":195},"Enterprise",{"href":196,"dataGaLocation":49,"dataGaName":197},"/enterprise/","enterprise",{"text":199,"config":200},"Small Business",{"href":201,"dataGaLocation":49,"dataGaName":202},"/small-business/","small business",{"text":204,"config":205},"Public Sector",{"href":206,"dataGaLocation":49,"dataGaName":207},"/solutions/public-sector/","public sector",{"text":209,"config":210},"Pricing",{"href":211,"dataGaName":212,"dataGaLocation":49,"dataNavLevelOne":212},"/pricing/","pricing",{"text":214,"config":215,"link":217,"lists":221,"feature":305},"Resources",{"dataNavLevelOne":216},"resources",{"text":218,"config":219},"View all resources",{"href":220,"dataGaName":216,"dataGaLocation":49},"/resources/",[222,255,277],{"title":223,"items":224},"Getting started",[225,230,235,240,245,250],{"text":226,"config":227},"Install",{"href":228,"dataGaName":229,"dataGaLocation":49},"/install/","install",{"text":231,"config":232},"Quick start guides",{"href":233,"dataGaName":234,"dataGaLocation":49},"/get-started/","quick setup checklists",{"text":236,"config":237},"Learn",{"href":238,"dataGaLocation":49,"dataGaName":239},"https://university.gitlab.com/","learn",{"text":241,"config":242},"Product documentation",{"href":243,"dataGaName":244,"dataGaLocation":49},"https://docs.gitlab.com/","product documentation",{"text":246,"config":247},"Best practice videos",{"href":248,"dataGaName":249,"dataGaLocation":49},"/getting-started-videos/","best practice videos",{"text":251,"config":252},"Integrations",{"href":253,"dataGaName":254,"dataGaLocation":49},"/integrations/","integrations",{"title":256,"items":257},"Discover",[258,263,267,272],{"text":259,"config":260},"Customer success stories",{"href":261,"dataGaName":262,"dataGaLocation":49},"/customers/","customer success stories",{"text":264,"config":265},"Blog",{"href":266,"dataGaName":5,"dataGaLocation":49},"/blog/",{"text":268,"config":269},"Remote",{"href":270,"dataGaName":271,"dataGaLocation":49},"https://handbook.gitlab.com/handbook/company/culture/all-remote/","remote",{"text":273,"config":274},"TeamOps",{"href":275,"dataGaName":276,"dataGaLocation":49},"/teamops/","teamops",{"title":278,"items":279},"Connect",[280,285,290,295,300],{"text":281,"config":282},"GitLab Services",{"href":283,"dataGaName":284,"dataGaLocation":49},"/services/","services",{"text":286,"config":287},"Community",{"href":288,"dataGaName":289,"dataGaLocation":49},"/community/","community",{"text":291,"config":292},"Forum",{"href":293,"dataGaName":294,"dataGaLocation":49},"https://forum.gitlab.com/","forum",{"text":296,"config":297},"Events",{"href":298,"dataGaName":299,"dataGaLocation":49},"/events/","events",{"text":301,"config":302},"Partners",{"href":303,"dataGaName":304,"dataGaLocation":49},"/partners/","partners",{"backgroundColor":306,"textColor":307,"text":308,"image":309,"link":313},"#2f2a6b","#fff","Insights for the future of software development",{"altText":310,"config":311},"the source promo card",{"src":312},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758208064/dzl0dbift9xdizyelkk4.svg",{"text":314,"config":315},"Read the latest",{"href":316,"dataGaName":317,"dataGaLocation":49},"/the-source/","the source",{"text":319,"config":320,"lists":322},"Company",{"dataNavLevelOne":321},"company",[323],{"items":324},[325,330,336,338,343,348,353,358,363,368,373],{"text":326,"config":327},"About",{"href":328,"dataGaName":329,"dataGaLocation":49},"/company/","about",{"text":331,"config":332,"footerGa":335},"Jobs",{"href":333,"dataGaName":334,"dataGaLocation":49},"/jobs/","jobs",{"dataGaName":334},{"text":296,"config":337},{"href":298,"dataGaName":299,"dataGaLocation":49},{"text":339,"config":340},"Leadership",{"href":341,"dataGaName":342,"dataGaLocation":49},"/company/team/e-group/","leadership",{"text":344,"config":345},"Team",{"href":346,"dataGaName":347,"dataGaLocation":49},"/company/team/","team",{"text":349,"config":350},"Handbook",{"href":351,"dataGaName":352,"dataGaLocation":49},"https://handbook.gitlab.com/","handbook",{"text":354,"config":355},"Investor relations",{"href":356,"dataGaName":357,"dataGaLocation":49},"https://ir.gitlab.com/","investor relations",{"text":359,"config":360},"Trust Center",{"href":361,"dataGaName":362,"dataGaLocation":49},"/security/","trust center",{"text":364,"config":365},"AI Transparency Center",{"href":366,"dataGaName":367,"dataGaLocation":49},"/ai-transparency-center/","ai transparency center",{"text":369,"config":370},"Newsletter",{"href":371,"dataGaName":372,"dataGaLocation":49},"/company/contact/","newsletter",{"text":374,"config":375},"Press",{"href":376,"dataGaName":377,"dataGaLocation":49},"/press/","press",{"text":379,"config":380,"lists":381},"Contact us",{"dataNavLevelOne":321},[382],{"items":383},[384,387,392],{"text":56,"config":385},{"href":58,"dataGaName":386,"dataGaLocation":49},"talk to sales",{"text":388,"config":389},"Get help",{"href":390,"dataGaName":391,"dataGaLocation":49},"/support/","get help",{"text":393,"config":394},"Customer portal",{"href":395,"dataGaName":396,"dataGaLocation":49},"https://customers.gitlab.com/customers/sign_in/","customer portal",{"close":398,"login":399,"suggestions":406},"Close",{"text":400,"link":401},"To search repositories and projects, login to",{"text":402,"config":403},"gitlab.com",{"href":63,"dataGaName":404,"dataGaLocation":405},"search login","search",{"text":407,"default":408},"Suggestions",[409,411,415,417,421,425],{"text":78,"config":410},{"href":83,"dataGaName":78,"dataGaLocation":405},{"text":412,"config":413},"Code Suggestions (AI)",{"href":414,"dataGaName":412,"dataGaLocation":405},"/solutions/code-suggestions/",{"text":130,"config":416},{"href":132,"dataGaName":130,"dataGaLocation":405},{"text":418,"config":419},"GitLab on AWS",{"href":420,"dataGaName":418,"dataGaLocation":405},"/partners/technology-partners/aws/",{"text":422,"config":423},"GitLab on Google Cloud",{"href":424,"dataGaName":422,"dataGaLocation":405},"/partners/technology-partners/google-cloud-platform/",{"text":426,"config":427},"Why GitLab?",{"href":91,"dataGaName":426,"dataGaLocation":405},{"freeTrial":429,"mobileIcon":434,"desktopIcon":439,"secondaryButton":442},{"text":430,"config":431},"Start free trial",{"href":432,"dataGaName":54,"dataGaLocation":433},"https://gitlab.com/-/trials/new/","nav",{"altText":435,"config":436},"Gitlab Icon",{"src":437,"dataGaName":438,"dataGaLocation":433},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203874/jypbw1jx72aexsoohd7x.svg","gitlab icon",{"altText":435,"config":440},{"src":441,"dataGaName":438,"dataGaLocation":433},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203875/gs4c8p8opsgvflgkswz9.svg",{"text":443,"config":444},"Get Started",{"href":445,"dataGaName":446,"dataGaLocation":433},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com/compare/gitlab-vs-github/","get started",{"freeTrial":448,"mobileIcon":452,"desktopIcon":454},{"text":449,"config":450},"Learn more about GitLab Duo",{"href":83,"dataGaName":451,"dataGaLocation":433},"gitlab duo",{"altText":435,"config":453},{"src":437,"dataGaName":438,"dataGaLocation":433},{"altText":435,"config":455},{"src":441,"dataGaName":438,"dataGaLocation":433},{"freeTrial":457,"mobileIcon":462,"desktopIcon":464},{"text":458,"config":459},"Back to pricing",{"href":211,"dataGaName":460,"dataGaLocation":433,"icon":461},"back to pricing","GoBack",{"altText":435,"config":463},{"src":437,"dataGaName":438,"dataGaLocation":433},{"altText":435,"config":465},{"src":441,"dataGaName":438,"dataGaLocation":433},"content:shared:en-us:main-navigation.yml","Main Navigation","shared/en-us/main-navigation.yml","shared/en-us/main-navigation",{"_path":471,"_dir":43,"_draft":6,"_partial":6,"_locale":7,"title":472,"button":473,"image":478,"config":482,"_id":484,"_type":35,"_source":37,"_file":485,"_stem":486,"_extension":40},"/shared/en-us/banner","is now in public beta!",{"text":474,"config":475},"Try the Beta",{"href":476,"dataGaName":477,"dataGaLocation":49},"/gitlab-duo/agent-platform/","duo banner",{"altText":479,"config":480},"GitLab Duo Agent Platform",{"src":481},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1753720689/somrf9zaunk0xlt7ne4x.svg",{"layout":483},"release","content:shared:en-us:banner.yml","shared/en-us/banner.yml","shared/en-us/banner",{"_path":488,"_dir":43,"_draft":6,"_partial":6,"_locale":7,"data":489,"_id":693,"_type":35,"title":694,"_source":37,"_file":695,"_stem":696,"_extension":40},"/shared/en-us/main-footer",{"text":490,"source":491,"edit":497,"contribute":502,"config":507,"items":512,"minimal":685},"Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license",{"text":492,"config":493},"View page source",{"href":494,"dataGaName":495,"dataGaLocation":496},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/","page source","footer",{"text":498,"config":499},"Edit this page",{"href":500,"dataGaName":501,"dataGaLocation":496},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/content/","web ide",{"text":503,"config":504},"Please contribute",{"href":505,"dataGaName":506,"dataGaLocation":496},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/CONTRIBUTING.md/","please contribute",{"twitter":508,"facebook":509,"youtube":510,"linkedin":511},"https://twitter.com/gitlab","https://www.facebook.com/gitlab","https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg","https://www.linkedin.com/company/gitlab-com",[513,536,592,621,655],{"title":67,"links":514,"subMenu":519},[515],{"text":516,"config":517},"DevSecOps platform",{"href":76,"dataGaName":518,"dataGaLocation":496},"devsecops platform",[520],{"title":209,"links":521},[522,526,531],{"text":523,"config":524},"View plans",{"href":211,"dataGaName":525,"dataGaLocation":496},"view plans",{"text":527,"config":528},"Why Premium?",{"href":529,"dataGaName":530,"dataGaLocation":496},"/pricing/premium/","why premium",{"text":532,"config":533},"Why Ultimate?",{"href":534,"dataGaName":535,"dataGaLocation":496},"/pricing/ultimate/","why ultimate",{"title":537,"links":538},"Solutions",[539,544,546,548,553,558,562,565,569,574,576,579,582,587],{"text":540,"config":541},"Digital transformation",{"href":542,"dataGaName":543,"dataGaLocation":496},"/topics/digital-transformation/","digital transformation",{"text":155,"config":545},{"href":157,"dataGaName":155,"dataGaLocation":496},{"text":144,"config":547},{"href":126,"dataGaName":127,"dataGaLocation":496},{"text":549,"config":550},"Agile development",{"href":551,"dataGaName":552,"dataGaLocation":496},"/solutions/agile-delivery/","agile delivery",{"text":554,"config":555},"Cloud transformation",{"href":556,"dataGaName":557,"dataGaLocation":496},"/topics/cloud-native/","cloud transformation",{"text":559,"config":560},"SCM",{"href":140,"dataGaName":561,"dataGaLocation":496},"source code management",{"text":130,"config":563},{"href":132,"dataGaName":564,"dataGaLocation":496},"continuous integration & delivery",{"text":566,"config":567},"Value stream management",{"href":184,"dataGaName":568,"dataGaLocation":496},"value stream management",{"text":570,"config":571},"GitOps",{"href":572,"dataGaName":573,"dataGaLocation":496},"/solutions/gitops/","gitops",{"text":194,"config":575},{"href":196,"dataGaName":197,"dataGaLocation":496},{"text":577,"config":578},"Small business",{"href":201,"dataGaName":202,"dataGaLocation":496},{"text":580,"config":581},"Public sector",{"href":206,"dataGaName":207,"dataGaLocation":496},{"text":583,"config":584},"Education",{"href":585,"dataGaName":586,"dataGaLocation":496},"/solutions/education/","education",{"text":588,"config":589},"Financial services",{"href":590,"dataGaName":591,"dataGaLocation":496},"/solutions/finance/","financial services",{"title":214,"links":593},[594,596,598,600,603,605,607,609,611,613,615,617,619],{"text":226,"config":595},{"href":228,"dataGaName":229,"dataGaLocation":496},{"text":231,"config":597},{"href":233,"dataGaName":234,"dataGaLocation":496},{"text":236,"config":599},{"href":238,"dataGaName":239,"dataGaLocation":496},{"text":241,"config":601},{"href":243,"dataGaName":602,"dataGaLocation":496},"docs",{"text":264,"config":604},{"href":266,"dataGaName":5,"dataGaLocation":496},{"text":259,"config":606},{"href":261,"dataGaName":262,"dataGaLocation":496},{"text":268,"config":608},{"href":270,"dataGaName":271,"dataGaLocation":496},{"text":281,"config":610},{"href":283,"dataGaName":284,"dataGaLocation":496},{"text":273,"config":612},{"href":275,"dataGaName":276,"dataGaLocation":496},{"text":286,"config":614},{"href":288,"dataGaName":289,"dataGaLocation":496},{"text":291,"config":616},{"href":293,"dataGaName":294,"dataGaLocation":496},{"text":296,"config":618},{"href":298,"dataGaName":299,"dataGaLocation":496},{"text":301,"config":620},{"href":303,"dataGaName":304,"dataGaLocation":496},{"title":319,"links":622},[623,625,627,629,631,633,635,639,644,646,648,650],{"text":326,"config":624},{"href":328,"dataGaName":321,"dataGaLocation":496},{"text":331,"config":626},{"href":333,"dataGaName":334,"dataGaLocation":496},{"text":339,"config":628},{"href":341,"dataGaName":342,"dataGaLocation":496},{"text":344,"config":630},{"href":346,"dataGaName":347,"dataGaLocation":496},{"text":349,"config":632},{"href":351,"dataGaName":352,"dataGaLocation":496},{"text":354,"config":634},{"href":356,"dataGaName":357,"dataGaLocation":496},{"text":636,"config":637},"Sustainability",{"href":638,"dataGaName":636,"dataGaLocation":496},"/sustainability/",{"text":640,"config":641},"Diversity, inclusion and belonging (DIB)",{"href":642,"dataGaName":643,"dataGaLocation":496},"/diversity-inclusion-belonging/","Diversity, inclusion and belonging",{"text":359,"config":645},{"href":361,"dataGaName":362,"dataGaLocation":496},{"text":369,"config":647},{"href":371,"dataGaName":372,"dataGaLocation":496},{"text":374,"config":649},{"href":376,"dataGaName":377,"dataGaLocation":496},{"text":651,"config":652},"Modern Slavery Transparency Statement",{"href":653,"dataGaName":654,"dataGaLocation":496},"https://handbook.gitlab.com/handbook/legal/modern-slavery-act-transparency-statement/","modern slavery transparency statement",{"title":656,"links":657},"Contact Us",[658,661,663,665,670,675,680],{"text":659,"config":660},"Contact an expert",{"href":58,"dataGaName":59,"dataGaLocation":496},{"text":388,"config":662},{"href":390,"dataGaName":391,"dataGaLocation":496},{"text":393,"config":664},{"href":395,"dataGaName":396,"dataGaLocation":496},{"text":666,"config":667},"Status",{"href":668,"dataGaName":669,"dataGaLocation":496},"https://status.gitlab.com/","status",{"text":671,"config":672},"Terms of use",{"href":673,"dataGaName":674,"dataGaLocation":496},"/terms/","terms of use",{"text":676,"config":677},"Privacy statement",{"href":678,"dataGaName":679,"dataGaLocation":496},"/privacy/","privacy statement",{"text":681,"config":682},"Cookie preferences",{"dataGaName":683,"dataGaLocation":496,"id":684,"isOneTrustButton":112},"cookie preferences","ot-sdk-btn",{"items":686},[687,689,691],{"text":671,"config":688},{"href":673,"dataGaName":674,"dataGaLocation":496},{"text":676,"config":690},{"href":678,"dataGaName":679,"dataGaLocation":496},{"text":681,"config":692},{"dataGaName":683,"dataGaLocation":496,"id":684,"isOneTrustButton":112},"content:shared:en-us:main-footer.yml","Main Footer","shared/en-us/main-footer.yml","shared/en-us/main-footer",[698,710,719,728,737,747],{"_path":699,"_dir":700,"_draft":6,"_partial":6,"_locale":7,"content":701,"config":705,"_id":707,"_type":35,"title":18,"_source":37,"_file":708,"_stem":709,"_extension":40},"/en-us/blog/authors/michael-henriksen","authors",{"name":18,"config":702},{"headshot":703,"ctfId":704},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749659488/Blog/Author%20Headshots/gitlab-logo-extra-whitespace.png","3DmojnawcJFqAgoNMCpFTX",{"template":706},"BlogAuthor","content:en-us:blog:authors:michael-henriksen.yml","en-us/blog/authors/michael-henriksen.yml","en-us/blog/authors/michael-henriksen",{"_path":711,"_dir":700,"_draft":6,"_partial":6,"_locale":7,"content":712,"config":715,"_id":716,"_type":35,"title":19,"_source":37,"_file":717,"_stem":718,"_extension":40},"/en-us/blog/authors/martynas-krupskis",{"name":19,"config":713},{"headshot":703,"ctfId":714},"3tK5S0f4QshGFGRrdEl7rn",{"template":706},"content:en-us:blog:authors:martynas-krupskis.yml","en-us/blog/authors/martynas-krupskis.yml","en-us/blog/authors/martynas-krupskis",{"_path":720,"_dir":700,"_draft":6,"_partial":6,"_locale":7,"content":721,"config":724,"_id":725,"_type":35,"title":20,"_source":37,"_file":726,"_stem":727,"_extension":40},"/en-us/blog/authors/mark-art",{"name":20,"config":722},{"headshot":703,"ctfId":723},"55KCfyNmgPaJRmBZhiN7k5",{"template":706},"content:en-us:blog:authors:mark-art.yml","en-us/blog/authors/mark-art.yml","en-us/blog/authors/mark-art",{"_path":729,"_dir":700,"_draft":6,"_partial":6,"_locale":7,"content":730,"config":733,"_id":734,"_type":35,"title":21,"_source":37,"_file":735,"_stem":736,"_extension":40},"/en-us/blog/authors/dinesh-bolkensteyn",{"name":21,"config":731},{"headshot":703,"ctfId":732},"EpylYWgjPmFOL5NX3Zxmk",{"template":706},"content:en-us:blog:authors:dinesh-bolkensteyn.yml","en-us/blog/authors/dinesh-bolkensteyn.yml","en-us/blog/authors/dinesh-bolkensteyn",{"_path":738,"_dir":700,"_draft":6,"_partial":6,"_locale":7,"content":739,"config":743,"_id":744,"_type":35,"title":22,"_source":37,"_file":745,"_stem":746,"_extension":40},"/en-us/blog/authors/isaac-dawson",{"name":22,"config":740},{"headshot":741,"ctfId":742},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749669814/Blog/Author%20Headshots/idawson-headshot.jpg","idawson",{"template":706},"content:en-us:blog:authors:isaac-dawson.yml","en-us/blog/authors/isaac-dawson.yml","en-us/blog/authors/isaac-dawson",{"_path":748,"_dir":700,"_draft":6,"_partial":6,"_locale":7,"content":749,"config":752,"_id":753,"_type":35,"title":23,"_source":37,"_file":754,"_stem":755,"_extension":40},"/en-us/blog/authors/julian-thome",{"name":23,"config":750},{"headshot":7,"ctfId":751},"jthome",{"template":706},"content:en-us:blog:authors:julian-thome.yml","en-us/blog/authors/julian-thome.yml","en-us/blog/authors/julian-thome",{"_path":757,"_dir":43,"_draft":6,"_partial":6,"_locale":7,"header":758,"eyebrow":759,"blurb":760,"button":761,"secondaryButton":765,"_id":767,"_type":35,"title":768,"_source":37,"_file":769,"_stem":770,"_extension":40},"/shared/en-us/next-steps","Start shipping better software faster","50%+ of the Fortune 100 trust GitLab","See what your team can do with the intelligent\n\n\nDevSecOps platform.\n",{"text":51,"config":762},{"href":763,"dataGaName":54,"dataGaLocation":764},"https://gitlab.com/-/trial_registrations/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/","feature",{"text":56,"config":766},{"href":58,"dataGaName":59,"dataGaLocation":764},"content:shared:en-us:next-steps.yml","Next Steps","shared/en-us/next-steps.yml","shared/en-us/next-steps",1758326240521]