[{"data":1,"prerenderedAt":730},["ShallowReactive",2],{"/en-us/blog/three-new-support-tools/":3,"navigation-en-us":37,"banner-en-us":466,"footer-en-us":483,"Will Chandler-Sara Kassabian":693,"next-steps-en-us":715},{"_path":4,"_dir":5,"_draft":6,"_partial":6,"_locale":7,"seo":8,"content":16,"config":27,"_id":30,"_type":31,"title":32,"_source":33,"_file":34,"_stem":35,"_extension":36},"/en-us/blog/three-new-support-tools","blog",false,"",{"title":9,"description":10,"ogTitle":9,"ogDescription":10,"noIndex":6,"ogImage":11,"ogUrl":12,"ogSiteName":13,"ogType":14,"canonicalUrls":12,"schema":15},"We've open sourced 3 tools to help troubleshoot system performance","Say hello to the open source tools our Support team is using to better summarize customer performance data – and find out how they can help you.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1749670405/Blog/Hero%20Images/open_source_tools.jpg","https://about.gitlab.com/blog/three-new-support-tools","https://about.gitlab.com","article","\n                        {\n        \"@context\": \"https://schema.org\",\n        \"@type\": \"Article\",\n        \"headline\": \"We've open sourced 3 tools to help troubleshoot system performance\",\n        \"author\": [{\"@type\":\"Person\",\"name\":\"Will Chandler\"},{\"@type\":\"Person\",\"name\":\"Sara Kassabian\"}],\n        \"datePublished\": \"2019-07-24\",\n      }",{"title":9,"description":10,"authors":17,"heroImage":11,"date":20,"body":21,"category":22,"tags":23},[18,19],"Will Chandler","Sara Kassabian","2019-07-24","Our self-managed customers often encounter issues related to performance, or\nthe time it takes to execute something. In the past, the [Support\nteam](/handbook/support/) had to pull data from disparate sources and cobble\nit together in order to analyze performance-related issues.\n\n\n“We’re dealing with someone else’s computer on support, so we have to be\nable to handle environments with limited observability,” says [Will\nChandler](/company/team/#wchandler), senior support engineer. “We’re at the\nmercy of their infrastructure. That’s why the team has made tools to reduce\nthe friction.”\n\n\n“With [GitLab.com](/pricing/), we have all of this fancy tooling that helps\nus collect performance data,” says [Lee Matos](/company/team/#leematos),\nsupport engineering manager. “But when we’re working with customers, we need\nto be ready to bring lightweight tools that don’t require a lot of setup\nthat we can use based on what they have in place.”\n\n\nThe Support team is working on becoming more data driven by using three new\ntools designed to aggregate and summarize performance data for self-managed\ncustomers. A focus on data-driven decision-making improves the customer\nrelationship and demonstrates our commitment to making performance a key\nfeature of GitLab.\n\n\nWe'll look at three open source tools created by GitLab Self-Managed\nSupport. Strace parser is a general tool that could be of use to anyone,\nwhile JSON Stats and GitLabSOS are tailored to GitLab, but could be easily\nmodified.\n\n\n## 1. [Strace\nparser](https://gitlab.com/gitlab-com/support/toolbox/strace-parser)\n\n\n[Strace](https://gitlab.com/strace/strace) is a commonly used debugging and\ndiagnostic tool in Linux that captures information about what’s happening\ninside processes running on our customers’ environments.\n\n\nUnlike [newer](http://man7.org/linux/man-pages/man1/perf.1.html) and [more\npowerful](https://github.com/iovisor/bpftrace) tracing tools, strace adds\n[significant overhead to a\nprocess](http://www.brendangregg.com/blog/2014-05-11/strace-wow-much-syscall.html).\nHowever, strace is generally available even on very old versions of Linux.\n\n\nAn strace of a single-threaded program is linear, but following the threads\nof execution quickly gets difficult when there are many processes being\ncaptured. At GitLab Support we are typically tracing\n[Unicorn](https://bogomips.org/unicorn/) workers or\n[Gitaly](https://gitlab.com/gitlab-org/gitaly), which are highly concurrent,\nresulting in hundreds of process IDs being traced and hundreds of thousands\nof lines of output from traces only a few seconds long.\n\n\nWill built [strace\nparser](https://gitlab.com/gitlab-com/support/toolbox/strace-parser) for\nthese types of use cases. Strace parser summarizes the most meaningful\nprocessing data delivered by an strace in a more accessible format, allowing\nusers to find the critical section sections of the data quickly.\n\n\nThe next two examples are from a GitLab customer that was using a very slow\nfile system to host their .gitconfig file, which was a major performance\nbottleneck. But it was not immediately clear what was happening from the\nperspective of a user trying to troubleshoot. By running an strace on\nGitaly, we were able to get a better understanding of why the system was so\nslow.\n\n\n```\n\n3694  13:45:06.207369 clock_gettime(CLOCK_MONOTONIC, {3016230, 201254200}) =\n0 \u003C0.000015>\n\n3694  13:45:06.207409 futex(0x7f645bb49664, FUTEX_WAIT_BITSET_PRIVATE,\n192398, {3016230, 299906871}, ffffffff \u003Cunfinished ...>\n\n3542  13:45:06.209616 \u003C... futex resumed> ) = -1 ETIMEDOUT (Connection timed\nout) \u003C0.005236>\n\n3542  13:45:06.209639 futex(0x1084ff0, FUTEX_WAKE, 1) = 1 \u003C0.000023>\n\n3510  13:45:06.209673 \u003C... futex resumed> ) = 0 \u003C0.002909>\n\n3542  13:45:06.209701 futex(0xc420896548, FUTEX_WAKE, 1 \u003Cunfinished ...>\n\n3510  13:45:06.209710 pselect6(0, NULL, NULL, NULL, {0, 20000}, NULL\n\u003Cunfinished ...>\n\n16780 13:45:06.209740 \u003C... futex resumed> ) = 0 \u003C0.002984>\n\n3542  13:45:06.209749 \u003C... futex resumed> ) = 1 \u003C0.000043>\n\n16780 13:45:06.209776 pselect6(0, NULL, NULL, NULL, {0, 3000}, NULL\n\u003Cunfinished ...>\n\n3542  13:45:06.209787 futex(0xc420053548, FUTEX_WAKE, 1 \u003Cunfinished ...>\n\n16780 13:45:06.209839 \u003C... pselect6 resumed> ) = 0 (Timeout) \u003C0.000056>\n\n3544  13:45:06.209853 \u003C... futex resumed> ) = 0 \u003C0.003148>\n\n3542  13:45:06.209861 \u003C... futex resumed> ) = 1 \u003C0.000069>\n\n3510  13:45:06.209868 \u003C... pselect6 resumed> ) = 0 (Timeout) \u003C0.000151>\n\n3544  13:45:06.209915 epoll_ctl(4\u003Canon_inode:[eventpoll]>, EPOLL_CTL_DEL,\n181\u003CUNIX:[164869291]>, 0xc42105bb14 \u003Cunfinished ...>\n\n16780 13:45:06.210076 write(1\u003Cpipe:[55447]>, \"time=\\\"2019-02-14T18:45:06Z\\\"\nlevel=warning msg=\\\"health check failed\\\" error=\\\"rpc error: code =\nDeadlineExceeded desc = context deadline exceeded\\\"\nworker.name=gitaly-ruby.4\\n\", 170 \u003Cunfinished ...>\n\n3544  13:45:06.210093 \u003C... epoll_ctl resumed> ) = 0 \u003C0.000053>\n\n3542  13:45:06.210101 futex(0x1089020, FUTEX_WAIT, 0, {0, 480025102}\n\u003Cunfinished ...>\n\n3510  13:45:06.210109 pselect6(0, NULL, NULL, NULL, {0, 20000}, NULL\n\u003Cunfinished ...>\n\n16780 13:45:06.210153 \u003C... write resumed> ) = 170 \u003C0.000064>\n\n3544  13:45:06.210163 close(181\u003CUNIX:[164869291]> \u003Cunfinished ...>\n\n```\n\n\nThis strace delivers more than 300,000 lines about the different Gitaly\nprocesses running on this customer’s GitLab environment, making it\nchallenging to decipher the flow of execution.\n\n{: .note.text-center}\n\n\n“In this case, we can use strace-parser to say, ‘Just give me all the files\nthat were opened, and sort them by how long it took to open,’” says Will.\n\n\n```\n\n$ strace-parser trace.txt files --sort duration\n\n\nFiles Opened\n\n      pid      dur (ms)       timestamp            error         file name\n  -------    ----------    ---------------    ---------------    ---------\n    24670      5203.999    13:45:16.152985           -           /efs/gitlab/home/.gitconfig\n    24859      5296.580    13:45:23.367482           -           /efs/gitlab/home/.gitconfig\n    24584      5279.810    13:45:09.286019           -           /efs/gitlab/home/.gitconfig\n    24666      5276.975    13:45:16.079697           -           /efs/gitlab/home/.gitconfig\n    24667      5255.649    13:45:16.101009           -           /efs/gitlab/home/.gitconfig\n    14871      2594.364    13:45:18.762347           -           /efs/gitlab/home/.gitconfig\n    24885      2440.635    13:45:26.224189           -           /efs/gitlab/home/.gitconfig\n    24886      2432.980    13:45:26.231009           -           /efs/gitlab/home/.gitconfig\n    24656        55.873    13:45:15.916836        ENOENT         /nfs/gitlab/gitdata/repositories/group/project.git/objects/info/alternates\n    24688        42.764    13:45:21.522789        ENOENT         /nfs/gitlab/gitdata/repositories/group/project.git/objects/info/alternates\n     3709        39.631    13:45:07.816618           -           /efs/gitlab/home/.gitconfig\n    24583        37.959    13:45:09.218283           -           /efs/gitlab/home/.gitconfig\n```\n\n\nBy summarizing the data in this way, we see multiple files that took 2-5\nseconds to open, which is several orders of magnitude slower than expected.\n\n{: .note.text-center}\n\n\n“If it’s a particularly busy server and we’re performing these actions 50\ntimes a second, 100 times a second, that adds up really fast,” says Will.\n“Strace-Parser lets you drill down quickly, and say, ‘OK, this specific\nthing we’re doing is super slow.’”\n\n\n### Get a closer look at processes using strace-parser\n\n\nStrace-Parser can also be used to drill down into details of a process.\n\n\nThe previous output showed PID 24670 is one of the slower processes, so we\nuse the parser to understand how this slow call impacted the performance of\nthe process overall.\n\n\n```\n\n$ strace-parser trace.txt pid 24670\n\n\nPID 24670\n\n  271 syscalls, active time: 5303.438ms, user time: 34.662ms, total time: 5338.100ms\n  start time: 13:45:16.116671    end time: 13:45:21.454771\n\n  syscall                 count    total (ms)      max (ms)      avg (ms)      min (ms)    errors\n  -----------------    --------    ----------    ----------    ----------    ----------    --------\n  open                       29      5223.073      5203.999       180.106         0.031    ENOENT: 9\n  read                       25        46.303        28.747         1.852         0.031\n  access                     11         6.948         4.131         0.632         0.056    ENOENT: 3\n  lstat                       6         5.116         2.130         0.853         0.077    ENOENT: 4\n  mmap                       32         3.868         0.485         0.121         0.028\n  openat                      2         3.757         2.934         1.878         0.823\n  fstat                      28         3.395         0.272         0.121         0.033\n  munmap                     11         2.551         0.929         0.232         0.056\n  rt_sigaction               59         2.548         0.121         0.043         0.024\n  close                      22         2.375         0.279         0.108         0.032\n  mprotect                   14         0.927         0.174         0.066         0.032\n  execve                      1         0.621         0.621         0.621         0.621\n  brk                         6         0.595         0.210         0.099         0.046\n  stat                        8         0.388         0.082         0.048         0.027    ENOENT: 3\n  getdents                    4         0.361         0.138         0.090         0.044\n  rt_sigprocmask              3         0.141         0.059         0.047         0.040\n  write                       1         0.101         0.101         0.101         0.101\n  dup2                        3         0.090         0.032         0.030         0.026\n  arch_prctl                  1         0.077         0.077         0.077         0.077\n  getrlimit                   1         0.062         0.062         0.062         0.062\n  getcwd                      1         0.044         0.044         0.044         0.044\n  set_robust_list             1         0.035         0.035         0.035         0.035\n  set_tid_address             1         0.032         0.032         0.032         0.032\n  setpgid                     1         0.030         0.030         0.030         0.030\n  ---------------\n\n  Program Executed: /opt/gitlab/embedded/bin/git\n  Args: [\"--git-dir\" \"/nfs/gitlab/gitdata/repositories/group/project.git\" \"cat-file\" \"--batch-check\"]\n\n  Parent PID:  3563\n\n  Slowest file open times for PID 24670:\n\n    dur (ms)       timestamp            error         file name\n  ----------    ---------------    ---------------    ---------\n    5203.999    13:45:16.152985           -           /efs/gitlab/home/.gitconfig\n       5.420    13:45:16.143520           -           /nfs/gitlab/gitdata/repositories/group/project.git/config\n       2.959    13:45:21.372776           -           /efs/gitlab/home/.gitconfig\n       2.934    13:45:21.401073           -           /nfs/gitlab/gitdata/repositories/group/project.git/refs/\n       2.736    13:45:21.417333        ENOENT         /nfs/gitlab/gitdata/repositories/group/project.git/info/grafts\n       2.683    13:45:21.421558           -           /nfs/gitlab/gitdata/repositories/group/project.git/objects/b7/ef5eba3a425af1e2a9cf6f51cb87454b6e1ad1\n       2.430    13:45:21.407170        ENOENT         /nfs/gitlab/gitdata/repositories/group/project.git/objects/info/alternates\n       0.992    13:45:21.420213        ENOENT         /nfs/gitlab/gitdata/repositories/group/project.git/shallow\n       0.823    13:45:21.405535           -           /nfs/gitlab/gitdata/repositories/group/project.git/objects/pack\n       0.275    13:45:21.380382           -           /nfs/gitlab/gitdata/repositories/group/project.git/config\n```\n\n\nThe output shows the time this process spent working was dominated by the\nslow file open. This data points the Support team in the right direction for\nfixing the underlying issue.\n\n{: .note.text-center}\n\n\nStrace itself has the `-c` flag which provides a similar summary, but its\nutility is limited when multiple processes are traced as it cannot break out\nper-process statistics.  Strace-Parser breaks these down to the PID level,\nand can also include the details of parent and child processes on demand.\n\n\n“In this case Will has identified an interesting area for our customer and\nthen very quickly anchored it in the fact that when we look at that one spot\nit was slow,” says Lee. “When we’re debugging, having this data available\nreally helps us pinpoint the problem for our customers so we can give them\nanswers.”\n\n\nThe typical GitLab deployment has many different processes and services\nrunning at a time, which can create dozens of different child processes, so\nthere is a large surface area for potential errors or slowness to occur.\n\n\nStrace-Parser is an open source, generic tool that anyone can use to better\nunderstand their strace data.\n\n\n## 2. [JSON Stats](https://gitlab.com/gitlab-com/support/toolbox/json_stats)\n\n\nWill also built [JSON\nStats](https://gitlab.com/gitlab-com/support/toolbox/json_stats), a script\nthat pulls performance statistics for different logs from the customer’s\nGitLab environment and summarizes the results in an easy-to-interpret table.\n\n\n```\n\nMETHOD                             COUNT     RPS     PERC99     PERC95    \nMEDIAN         MAX        MIN          SCORE    % FAIL\n\nFetchRemote                         2542    0.17  962176.08  130154.88  \n36580.23  4988513.00    1940.45  2445851585.19      1.06\n\nFindAllTags                         5200    0.34   30000.37   11538.63   \n1941.84    30006.23     252.10   156001924.68      1.63\n\nFindCommit                          3506    0.23   20859.98   16622.78  \n10841.86    30001.59    2528.67    73135073.75      0.23\n\nFindAllRemoteBranches               1664    0.11   20432.93   12996.75   \n8606.60   405503.94    1430.84    34000396.10      0.00\n\nAddRemote                           2603    0.17   10001.03    8094.97    \n825.46    10007.46     228.13    26032673.70      3.00\n\nFindLocalBranches                   2535    0.16   10004.68   10002.90   \n9051.91    10036.16    1260.89    25361871.05     34.32\n\n```\n\n\nThis output shows that we’re calling the “FindLocalBranches” service 2500+\ntimes, and it’s failing 34% of the time.\n\n{: .note.text-center}\n\n\nThe Support team can use JSON Stats to ground their findings in evidence\nwhen evaluating overall performance for a customer. It's the same concept as\nstrace-parser. Can we pivot the information in a way that it clearly becomes\nmeaningful data?\n\n\n“It’s a quick way of extracting data that you can give to a customer.\nInstead of saying ‘Look, this failed once,’ we can say, ‘Look, this is\nfailing a third of the time and that suggests there’s a problem with X,’”\nsays Will.\n\n\nIn the sample output we see that JSON Stats is working with Gitaly logs, but\nthe tool is nimble enough to work on the logs from all the heavy components\nof GitLab, including Rails, which runs the UI, and Sidekiq, which works on\nbackground tasks.\n\n\n“Some of our customers are very sophisticated and may have advanced\nmonitoring that could give us this information. But we wanted to build a\ntool that would help us align and easily standardize on how we can get this\nperformance information for customers that don’t have an advanced monitoring\nsetup,” says Lee.\n\n\nWhile this specific tool isn't as helpful for people outside of the GitLab\ncommunity, hopefully it helps to inspire others to consider how they are\ndrawing conclusions, and how they can speed that process up.\n\n\n### Benchmarking with JSON Stats\n\n\nWill is building a future iteration of JSON Stats that will compare the\nperformance of a customer’s GitLab instance with GitLab.com.\n\n\n![JSON benchmarking\ntable](https://about.gitlab.com/images/blogimages/support-tools-update.png){:\n.shadow}\n\n\nBenchmarking the performance of GitLab.com (the first row) with the customer\nenvironment (second row), and the ratio between the two (third row). We can\nsee that in the worst case, the customer’s 99th percentile FindCommit\nlatency was almost eight times slower than it was on GitLab.com.\n\n{: .note.text-center}\n\n\n“Our vision here is to give accountability to our customers. We’re going to\ntreat GitLab.com as the pinnacle experience for GitLab,” says Lee. “We want\nto use JSON Stats with benchmarking to help us understand how far away our\ncustomers are from GitLab.com.”\n\n\nLee and Will are still assessing how to set the target range for the\ncustomer’s instance of GitLab. But considering the wealth of resources\nallocated to GitLab.com, any self-managed customer that is performing within\n5-10% of GitLab.com would be considered hugely successful.\n\n\n## 3. [GitLab SOS](https://gitlab.com/gitlab-com/support/toolbox/gitlabsos)\n\n\nWhen a customer encounters an issue, but they are unsure of what they\nproblem is, they can run [GitLab\nSOS](https://gitlab.com/gitlab-com/support/toolbox/gitlabsos), created by\nsupport engineer [Cody West](/company/team/#codyww), to create a snapshot of\ndifferent activities happening on their system. It's been so helpful in\ndebugging GitLab that it's being added into our [Omnibus\ndelivery](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/3430).\n\n\nBy capturing so much data about a moment in time during or shortly after\nencountering a problem, the support team is able to work asynchronously to\ntroubleshoot on behalf of the customer.\n\n\n```\n\ncpuinfo              getenforce           iotop               \nnetstat              opt                  sestatus             unicorn_stats\n\ndf_h                 gitlab_status        lscpu               \nnetstat_i            pidstat              systemctl_unit_files uptime\n\ndmesg                gitlabsos.log        meminfo             \nnfsiostat            ps                   tainted              var\n\netc                  hostname             mount               \nnfsstat              sar_dev              ulimit               vmstat\n\nfree_m               iostat               mpstat              \nntpq                 sar_tcp              uname\n\n```\n\n\nGitLab SOS works best if the script is run while an issue is occurring, or\nmoments after, but even if the window of opportunity is missed you can still\nsuccessfully gather information to diagnose the problem.\n\n{: .note.text-center}\n\n\n“If a customer is sharp, they may know what problems to look for already,”\nsays Lee. “But if a customer is scared and they don’t know what to look for,\nthen they can lean on a tool like GitLab SOS and learn from GitLab SOS. We\neven have some sharp customers that will generate the SOS output and begin\nto troubleshoot themselves because of the comprehensive overview it\nprovides.”\n\n\n## These new tools drive data-driven decision-making in Support\n\n\nTools like strace-parser, JSON Stats, and GitLab SOS provide the Support\nteam and GitLab customers with critical evidence about performance. By\nletting the data drive decision-making, the Support team is able to identify\nproblems faster and quickly start debugging customer environments.\nPerformance is a key feature of GitLab, and by filling our toolbox with\ndata-driven solutions we can ensure greater\n[transparency](https://handbook.gitlab.com/handbook/values/#transparency) between GitLab and our\ncustomers.\n\n\nLearn more about debugging from a support engineering perspective in a\nGitLab Unfiltered video.\n\n\n\u003Cfigure class=\"video_container\">\n  \u003Ciframe src=\"https://www.youtube.com/embed/9W6QnpYewik\" frameborder=\"0\" allowfullscreen=\"true\"> \u003C/iframe>\n\u003C/figure>\n\n\nCover photo by [Diogo\nNunes](https://unsplash.com/@dialex?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText)\non\n[Unsplash](https://unsplash.com/search/photos/tools?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText)\n\n{: .note}\n","engineering",[24,25,26],"open source","features","inside GitLab",{"slug":28,"featured":6,"template":29},"three-new-support-tools","BlogPost","content:en-us:blog:three-new-support-tools.yml","yaml","Three New Support Tools","content","en-us/blog/three-new-support-tools.yml","en-us/blog/three-new-support-tools","yml",{"_path":38,"_dir":39,"_draft":6,"_partial":6,"_locale":7,"data":40,"_id":462,"_type":31,"title":463,"_source":33,"_file":464,"_stem":465,"_extension":36},"/shared/en-us/main-navigation","en-us",{"logo":41,"freeTrial":46,"sales":51,"login":56,"items":61,"search":393,"minimal":424,"duo":443,"pricingDeployment":452},{"config":42},{"href":43,"dataGaName":44,"dataGaLocation":45},"/","gitlab logo","header",{"text":47,"config":48},"Get free trial",{"href":49,"dataGaName":50,"dataGaLocation":45},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com&glm_content=default-saas-trial/","free trial",{"text":52,"config":53},"Talk to sales",{"href":54,"dataGaName":55,"dataGaLocation":45},"/sales/","sales",{"text":57,"config":58},"Sign in",{"href":59,"dataGaName":60,"dataGaLocation":45},"https://gitlab.com/users/sign_in/","sign in",[62,106,204,209,314,374],{"text":63,"config":64,"cards":66,"footer":89},"Platform",{"dataNavLevelOne":65},"platform",[67,73,81],{"title":63,"description":68,"link":69},"The most comprehensive AI-powered DevSecOps Platform",{"text":70,"config":71},"Explore our Platform",{"href":72,"dataGaName":65,"dataGaLocation":45},"/platform/",{"title":74,"description":75,"link":76},"GitLab Duo (AI)","Build software faster with AI at every stage of development",{"text":77,"config":78},"Meet GitLab Duo",{"href":79,"dataGaName":80,"dataGaLocation":45},"/gitlab-duo/","gitlab duo ai",{"title":82,"description":83,"link":84},"Why GitLab","10 reasons why Enterprises choose GitLab",{"text":85,"config":86},"Learn more",{"href":87,"dataGaName":88,"dataGaLocation":45},"/why-gitlab/","why gitlab",{"title":90,"items":91},"Get started with",[92,97,102],{"text":93,"config":94},"Platform Engineering",{"href":95,"dataGaName":96,"dataGaLocation":45},"/solutions/platform-engineering/","platform engineering",{"text":98,"config":99},"Developer Experience",{"href":100,"dataGaName":101,"dataGaLocation":45},"/developer-experience/","Developer experience",{"text":103,"config":104},"MLOps",{"href":105,"dataGaName":103,"dataGaLocation":45},"/topics/devops/the-role-of-ai-in-devops/",{"text":107,"left":108,"config":109,"link":111,"lists":115,"footer":186},"Product",true,{"dataNavLevelOne":110},"solutions",{"text":112,"config":113},"View all Solutions",{"href":114,"dataGaName":110,"dataGaLocation":45},"/solutions/",[116,141,165],{"title":117,"description":118,"link":119,"items":124},"Automation","CI/CD and automation to accelerate deployment",{"config":120},{"icon":121,"href":122,"dataGaName":123,"dataGaLocation":45},"AutomatedCodeAlt","/solutions/delivery-automation/","automated software delivery",[125,129,133,137],{"text":126,"config":127},"CI/CD",{"href":128,"dataGaLocation":45,"dataGaName":126},"/solutions/continuous-integration/",{"text":130,"config":131},"AI-Assisted Development",{"href":79,"dataGaLocation":45,"dataGaName":132},"AI assisted development",{"text":134,"config":135},"Source Code Management",{"href":136,"dataGaLocation":45,"dataGaName":134},"/solutions/source-code-management/",{"text":138,"config":139},"Automated Software Delivery",{"href":122,"dataGaLocation":45,"dataGaName":140},"Automated software delivery",{"title":142,"description":143,"link":144,"items":149},"Security","Deliver code faster without compromising security",{"config":145},{"href":146,"dataGaName":147,"dataGaLocation":45,"icon":148},"/solutions/security-compliance/","security and compliance","ShieldCheckLight",[150,155,160],{"text":151,"config":152},"Application Security Testing",{"href":153,"dataGaName":154,"dataGaLocation":45},"/solutions/application-security-testing/","Application security testing",{"text":156,"config":157},"Software Supply Chain Security",{"href":158,"dataGaLocation":45,"dataGaName":159},"/solutions/supply-chain/","Software supply chain security",{"text":161,"config":162},"Software Compliance",{"href":163,"dataGaName":164,"dataGaLocation":45},"/solutions/software-compliance/","software compliance",{"title":166,"link":167,"items":172},"Measurement",{"config":168},{"icon":169,"href":170,"dataGaName":171,"dataGaLocation":45},"DigitalTransformation","/solutions/visibility-measurement/","visibility and measurement",[173,177,181],{"text":174,"config":175},"Visibility & Measurement",{"href":170,"dataGaLocation":45,"dataGaName":176},"Visibility and Measurement",{"text":178,"config":179},"Value Stream Management",{"href":180,"dataGaLocation":45,"dataGaName":178},"/solutions/value-stream-management/",{"text":182,"config":183},"Analytics & Insights",{"href":184,"dataGaLocation":45,"dataGaName":185},"/solutions/analytics-and-insights/","Analytics and insights",{"title":187,"items":188},"GitLab for",[189,194,199],{"text":190,"config":191},"Enterprise",{"href":192,"dataGaLocation":45,"dataGaName":193},"/enterprise/","enterprise",{"text":195,"config":196},"Small Business",{"href":197,"dataGaLocation":45,"dataGaName":198},"/small-business/","small business",{"text":200,"config":201},"Public Sector",{"href":202,"dataGaLocation":45,"dataGaName":203},"/solutions/public-sector/","public sector",{"text":205,"config":206},"Pricing",{"href":207,"dataGaName":208,"dataGaLocation":45,"dataNavLevelOne":208},"/pricing/","pricing",{"text":210,"config":211,"link":213,"lists":217,"feature":301},"Resources",{"dataNavLevelOne":212},"resources",{"text":214,"config":215},"View all resources",{"href":216,"dataGaName":212,"dataGaLocation":45},"/resources/",[218,251,273],{"title":219,"items":220},"Getting started",[221,226,231,236,241,246],{"text":222,"config":223},"Install",{"href":224,"dataGaName":225,"dataGaLocation":45},"/install/","install",{"text":227,"config":228},"Quick start guides",{"href":229,"dataGaName":230,"dataGaLocation":45},"/get-started/","quick setup checklists",{"text":232,"config":233},"Learn",{"href":234,"dataGaLocation":45,"dataGaName":235},"https://university.gitlab.com/","learn",{"text":237,"config":238},"Product documentation",{"href":239,"dataGaName":240,"dataGaLocation":45},"https://docs.gitlab.com/","product documentation",{"text":242,"config":243},"Best practice videos",{"href":244,"dataGaName":245,"dataGaLocation":45},"/getting-started-videos/","best practice videos",{"text":247,"config":248},"Integrations",{"href":249,"dataGaName":250,"dataGaLocation":45},"/integrations/","integrations",{"title":252,"items":253},"Discover",[254,259,263,268],{"text":255,"config":256},"Customer success stories",{"href":257,"dataGaName":258,"dataGaLocation":45},"/customers/","customer success stories",{"text":260,"config":261},"Blog",{"href":262,"dataGaName":5,"dataGaLocation":45},"/blog/",{"text":264,"config":265},"Remote",{"href":266,"dataGaName":267,"dataGaLocation":45},"https://handbook.gitlab.com/handbook/company/culture/all-remote/","remote",{"text":269,"config":270},"TeamOps",{"href":271,"dataGaName":272,"dataGaLocation":45},"/teamops/","teamops",{"title":274,"items":275},"Connect",[276,281,286,291,296],{"text":277,"config":278},"GitLab Services",{"href":279,"dataGaName":280,"dataGaLocation":45},"/services/","services",{"text":282,"config":283},"Community",{"href":284,"dataGaName":285,"dataGaLocation":45},"/community/","community",{"text":287,"config":288},"Forum",{"href":289,"dataGaName":290,"dataGaLocation":45},"https://forum.gitlab.com/","forum",{"text":292,"config":293},"Events",{"href":294,"dataGaName":295,"dataGaLocation":45},"/events/","events",{"text":297,"config":298},"Partners",{"href":299,"dataGaName":300,"dataGaLocation":45},"/partners/","partners",{"backgroundColor":302,"textColor":303,"text":304,"image":305,"link":309},"#2f2a6b","#fff","Insights for the future of software development",{"altText":306,"config":307},"the source promo card",{"src":308},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758208064/dzl0dbift9xdizyelkk4.svg",{"text":310,"config":311},"Read the latest",{"href":312,"dataGaName":313,"dataGaLocation":45},"/the-source/","the source",{"text":315,"config":316,"lists":318},"Company",{"dataNavLevelOne":317},"company",[319],{"items":320},[321,326,332,334,339,344,349,354,359,364,369],{"text":322,"config":323},"About",{"href":324,"dataGaName":325,"dataGaLocation":45},"/company/","about",{"text":327,"config":328,"footerGa":331},"Jobs",{"href":329,"dataGaName":330,"dataGaLocation":45},"/jobs/","jobs",{"dataGaName":330},{"text":292,"config":333},{"href":294,"dataGaName":295,"dataGaLocation":45},{"text":335,"config":336},"Leadership",{"href":337,"dataGaName":338,"dataGaLocation":45},"/company/team/e-group/","leadership",{"text":340,"config":341},"Team",{"href":342,"dataGaName":343,"dataGaLocation":45},"/company/team/","team",{"text":345,"config":346},"Handbook",{"href":347,"dataGaName":348,"dataGaLocation":45},"https://handbook.gitlab.com/","handbook",{"text":350,"config":351},"Investor relations",{"href":352,"dataGaName":353,"dataGaLocation":45},"https://ir.gitlab.com/","investor relations",{"text":355,"config":356},"Trust Center",{"href":357,"dataGaName":358,"dataGaLocation":45},"/security/","trust center",{"text":360,"config":361},"AI Transparency Center",{"href":362,"dataGaName":363,"dataGaLocation":45},"/ai-transparency-center/","ai transparency center",{"text":365,"config":366},"Newsletter",{"href":367,"dataGaName":368,"dataGaLocation":45},"/company/contact/","newsletter",{"text":370,"config":371},"Press",{"href":372,"dataGaName":373,"dataGaLocation":45},"/press/","press",{"text":375,"config":376,"lists":377},"Contact us",{"dataNavLevelOne":317},[378],{"items":379},[380,383,388],{"text":52,"config":381},{"href":54,"dataGaName":382,"dataGaLocation":45},"talk to sales",{"text":384,"config":385},"Get help",{"href":386,"dataGaName":387,"dataGaLocation":45},"/support/","get help",{"text":389,"config":390},"Customer portal",{"href":391,"dataGaName":392,"dataGaLocation":45},"https://customers.gitlab.com/customers/sign_in/","customer portal",{"close":394,"login":395,"suggestions":402},"Close",{"text":396,"link":397},"To search repositories and projects, login to",{"text":398,"config":399},"gitlab.com",{"href":59,"dataGaName":400,"dataGaLocation":401},"search login","search",{"text":403,"default":404},"Suggestions",[405,407,411,413,417,421],{"text":74,"config":406},{"href":79,"dataGaName":74,"dataGaLocation":401},{"text":408,"config":409},"Code Suggestions (AI)",{"href":410,"dataGaName":408,"dataGaLocation":401},"/solutions/code-suggestions/",{"text":126,"config":412},{"href":128,"dataGaName":126,"dataGaLocation":401},{"text":414,"config":415},"GitLab on AWS",{"href":416,"dataGaName":414,"dataGaLocation":401},"/partners/technology-partners/aws/",{"text":418,"config":419},"GitLab on Google Cloud",{"href":420,"dataGaName":418,"dataGaLocation":401},"/partners/technology-partners/google-cloud-platform/",{"text":422,"config":423},"Why GitLab?",{"href":87,"dataGaName":422,"dataGaLocation":401},{"freeTrial":425,"mobileIcon":430,"desktopIcon":435,"secondaryButton":438},{"text":426,"config":427},"Start free trial",{"href":428,"dataGaName":50,"dataGaLocation":429},"https://gitlab.com/-/trials/new/","nav",{"altText":431,"config":432},"Gitlab Icon",{"src":433,"dataGaName":434,"dataGaLocation":429},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203874/jypbw1jx72aexsoohd7x.svg","gitlab icon",{"altText":431,"config":436},{"src":437,"dataGaName":434,"dataGaLocation":429},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203875/gs4c8p8opsgvflgkswz9.svg",{"text":439,"config":440},"Get Started",{"href":441,"dataGaName":442,"dataGaLocation":429},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com/compare/gitlab-vs-github/","get started",{"freeTrial":444,"mobileIcon":448,"desktopIcon":450},{"text":445,"config":446},"Learn more about GitLab Duo",{"href":79,"dataGaName":447,"dataGaLocation":429},"gitlab duo",{"altText":431,"config":449},{"src":433,"dataGaName":434,"dataGaLocation":429},{"altText":431,"config":451},{"src":437,"dataGaName":434,"dataGaLocation":429},{"freeTrial":453,"mobileIcon":458,"desktopIcon":460},{"text":454,"config":455},"Back to pricing",{"href":207,"dataGaName":456,"dataGaLocation":429,"icon":457},"back to pricing","GoBack",{"altText":431,"config":459},{"src":433,"dataGaName":434,"dataGaLocation":429},{"altText":431,"config":461},{"src":437,"dataGaName":434,"dataGaLocation":429},"content:shared:en-us:main-navigation.yml","Main Navigation","shared/en-us/main-navigation.yml","shared/en-us/main-navigation",{"_path":467,"_dir":39,"_draft":6,"_partial":6,"_locale":7,"title":468,"button":469,"image":474,"config":478,"_id":480,"_type":31,"_source":33,"_file":481,"_stem":482,"_extension":36},"/shared/en-us/banner","is now in public beta!",{"text":470,"config":471},"Try the Beta",{"href":472,"dataGaName":473,"dataGaLocation":45},"/gitlab-duo/agent-platform/","duo banner",{"altText":475,"config":476},"GitLab Duo Agent Platform",{"src":477},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1753720689/somrf9zaunk0xlt7ne4x.svg",{"layout":479},"release","content:shared:en-us:banner.yml","shared/en-us/banner.yml","shared/en-us/banner",{"_path":484,"_dir":39,"_draft":6,"_partial":6,"_locale":7,"data":485,"_id":689,"_type":31,"title":690,"_source":33,"_file":691,"_stem":692,"_extension":36},"/shared/en-us/main-footer",{"text":486,"source":487,"edit":493,"contribute":498,"config":503,"items":508,"minimal":681},"Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license",{"text":488,"config":489},"View page source",{"href":490,"dataGaName":491,"dataGaLocation":492},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/","page source","footer",{"text":494,"config":495},"Edit this page",{"href":496,"dataGaName":497,"dataGaLocation":492},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/content/","web ide",{"text":499,"config":500},"Please contribute",{"href":501,"dataGaName":502,"dataGaLocation":492},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/CONTRIBUTING.md/","please contribute",{"twitter":504,"facebook":505,"youtube":506,"linkedin":507},"https://twitter.com/gitlab","https://www.facebook.com/gitlab","https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg","https://www.linkedin.com/company/gitlab-com",[509,532,588,617,651],{"title":63,"links":510,"subMenu":515},[511],{"text":512,"config":513},"DevSecOps platform",{"href":72,"dataGaName":514,"dataGaLocation":492},"devsecops platform",[516],{"title":205,"links":517},[518,522,527],{"text":519,"config":520},"View plans",{"href":207,"dataGaName":521,"dataGaLocation":492},"view plans",{"text":523,"config":524},"Why Premium?",{"href":525,"dataGaName":526,"dataGaLocation":492},"/pricing/premium/","why premium",{"text":528,"config":529},"Why Ultimate?",{"href":530,"dataGaName":531,"dataGaLocation":492},"/pricing/ultimate/","why ultimate",{"title":533,"links":534},"Solutions",[535,540,542,544,549,554,558,561,565,570,572,575,578,583],{"text":536,"config":537},"Digital transformation",{"href":538,"dataGaName":539,"dataGaLocation":492},"/topics/digital-transformation/","digital transformation",{"text":151,"config":541},{"href":153,"dataGaName":151,"dataGaLocation":492},{"text":140,"config":543},{"href":122,"dataGaName":123,"dataGaLocation":492},{"text":545,"config":546},"Agile development",{"href":547,"dataGaName":548,"dataGaLocation":492},"/solutions/agile-delivery/","agile delivery",{"text":550,"config":551},"Cloud transformation",{"href":552,"dataGaName":553,"dataGaLocation":492},"/topics/cloud-native/","cloud transformation",{"text":555,"config":556},"SCM",{"href":136,"dataGaName":557,"dataGaLocation":492},"source code management",{"text":126,"config":559},{"href":128,"dataGaName":560,"dataGaLocation":492},"continuous integration & delivery",{"text":562,"config":563},"Value stream management",{"href":180,"dataGaName":564,"dataGaLocation":492},"value stream management",{"text":566,"config":567},"GitOps",{"href":568,"dataGaName":569,"dataGaLocation":492},"/solutions/gitops/","gitops",{"text":190,"config":571},{"href":192,"dataGaName":193,"dataGaLocation":492},{"text":573,"config":574},"Small business",{"href":197,"dataGaName":198,"dataGaLocation":492},{"text":576,"config":577},"Public sector",{"href":202,"dataGaName":203,"dataGaLocation":492},{"text":579,"config":580},"Education",{"href":581,"dataGaName":582,"dataGaLocation":492},"/solutions/education/","education",{"text":584,"config":585},"Financial services",{"href":586,"dataGaName":587,"dataGaLocation":492},"/solutions/finance/","financial services",{"title":210,"links":589},[590,592,594,596,599,601,603,605,607,609,611,613,615],{"text":222,"config":591},{"href":224,"dataGaName":225,"dataGaLocation":492},{"text":227,"config":593},{"href":229,"dataGaName":230,"dataGaLocation":492},{"text":232,"config":595},{"href":234,"dataGaName":235,"dataGaLocation":492},{"text":237,"config":597},{"href":239,"dataGaName":598,"dataGaLocation":492},"docs",{"text":260,"config":600},{"href":262,"dataGaName":5,"dataGaLocation":492},{"text":255,"config":602},{"href":257,"dataGaName":258,"dataGaLocation":492},{"text":264,"config":604},{"href":266,"dataGaName":267,"dataGaLocation":492},{"text":277,"config":606},{"href":279,"dataGaName":280,"dataGaLocation":492},{"text":269,"config":608},{"href":271,"dataGaName":272,"dataGaLocation":492},{"text":282,"config":610},{"href":284,"dataGaName":285,"dataGaLocation":492},{"text":287,"config":612},{"href":289,"dataGaName":290,"dataGaLocation":492},{"text":292,"config":614},{"href":294,"dataGaName":295,"dataGaLocation":492},{"text":297,"config":616},{"href":299,"dataGaName":300,"dataGaLocation":492},{"title":315,"links":618},[619,621,623,625,627,629,631,635,640,642,644,646],{"text":322,"config":620},{"href":324,"dataGaName":317,"dataGaLocation":492},{"text":327,"config":622},{"href":329,"dataGaName":330,"dataGaLocation":492},{"text":335,"config":624},{"href":337,"dataGaName":338,"dataGaLocation":492},{"text":340,"config":626},{"href":342,"dataGaName":343,"dataGaLocation":492},{"text":345,"config":628},{"href":347,"dataGaName":348,"dataGaLocation":492},{"text":350,"config":630},{"href":352,"dataGaName":353,"dataGaLocation":492},{"text":632,"config":633},"Sustainability",{"href":634,"dataGaName":632,"dataGaLocation":492},"/sustainability/",{"text":636,"config":637},"Diversity, inclusion and belonging (DIB)",{"href":638,"dataGaName":639,"dataGaLocation":492},"/diversity-inclusion-belonging/","Diversity, inclusion and belonging",{"text":355,"config":641},{"href":357,"dataGaName":358,"dataGaLocation":492},{"text":365,"config":643},{"href":367,"dataGaName":368,"dataGaLocation":492},{"text":370,"config":645},{"href":372,"dataGaName":373,"dataGaLocation":492},{"text":647,"config":648},"Modern Slavery Transparency Statement",{"href":649,"dataGaName":650,"dataGaLocation":492},"https://handbook.gitlab.com/handbook/legal/modern-slavery-act-transparency-statement/","modern slavery transparency statement",{"title":652,"links":653},"Contact Us",[654,657,659,661,666,671,676],{"text":655,"config":656},"Contact an expert",{"href":54,"dataGaName":55,"dataGaLocation":492},{"text":384,"config":658},{"href":386,"dataGaName":387,"dataGaLocation":492},{"text":389,"config":660},{"href":391,"dataGaName":392,"dataGaLocation":492},{"text":662,"config":663},"Status",{"href":664,"dataGaName":665,"dataGaLocation":492},"https://status.gitlab.com/","status",{"text":667,"config":668},"Terms of use",{"href":669,"dataGaName":670,"dataGaLocation":492},"/terms/","terms of use",{"text":672,"config":673},"Privacy statement",{"href":674,"dataGaName":675,"dataGaLocation":492},"/privacy/","privacy statement",{"text":677,"config":678},"Cookie preferences",{"dataGaName":679,"dataGaLocation":492,"id":680,"isOneTrustButton":108},"cookie preferences","ot-sdk-btn",{"items":682},[683,685,687],{"text":667,"config":684},{"href":669,"dataGaName":670,"dataGaLocation":492},{"text":672,"config":686},{"href":674,"dataGaName":675,"dataGaLocation":492},{"text":677,"config":688},{"dataGaName":679,"dataGaLocation":492,"id":680,"isOneTrustButton":108},"content:shared:en-us:main-footer.yml","Main Footer","shared/en-us/main-footer.yml","shared/en-us/main-footer",[694,706],{"_path":695,"_dir":696,"_draft":6,"_partial":6,"_locale":7,"content":697,"config":701,"_id":703,"_type":31,"title":18,"_source":33,"_file":704,"_stem":705,"_extension":36},"/en-us/blog/authors/will-chandler","authors",{"name":18,"config":698},{"headshot":699,"ctfId":700},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749659488/Blog/Author%20Headshots/gitlab-logo-extra-whitespace.png","DKiIGSSRIyO6QdTQkRkjs",{"template":702},"BlogAuthor","content:en-us:blog:authors:will-chandler.yml","en-us/blog/authors/will-chandler.yml","en-us/blog/authors/will-chandler",{"_path":707,"_dir":696,"_draft":6,"_partial":6,"_locale":7,"content":708,"config":711,"_id":712,"_type":31,"title":19,"_source":33,"_file":713,"_stem":714,"_extension":36},"/en-us/blog/authors/sara-kassabian",{"name":19,"config":709},{"headshot":7,"ctfId":710},"skassabian",{"template":702},"content:en-us:blog:authors:sara-kassabian.yml","en-us/blog/authors/sara-kassabian.yml","en-us/blog/authors/sara-kassabian",{"_path":716,"_dir":39,"_draft":6,"_partial":6,"_locale":7,"header":717,"eyebrow":718,"blurb":719,"button":720,"secondaryButton":724,"_id":726,"_type":31,"title":727,"_source":33,"_file":728,"_stem":729,"_extension":36},"/shared/en-us/next-steps","Start shipping better software faster","50%+ of the Fortune 100 trust GitLab","See what your team can do with the intelligent\n\n\nDevSecOps platform.\n",{"text":47,"config":721},{"href":722,"dataGaName":50,"dataGaLocation":723},"https://gitlab.com/-/trial_registrations/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/","feature",{"text":52,"config":725},{"href":54,"dataGaName":55,"dataGaLocation":723},"content:shared:en-us:next-steps.yml","Next Steps","shared/en-us/next-steps.yml","shared/en-us/next-steps",1758326246162]