Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Site reliability engineering
open-in-new
<h2 id="history">History</h2> <p>Site Reliability Engineering originated at <a href="/facts/Google/GT9Sugza">Google</a> with Benjamin Treynor Sloss,<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a><a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a> who founded SRE team in 2003.<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a> The concept expanded within the <a href="/facts/Software_development/UUfsAaYY">software development</a> industry, leading various companies to employ site <a href="/facts/Reliability_engineering/cd6AS5w3">reliability engineers</a>.<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a> By March 2016, Google had more than 1,000 site reliability engineers on staff.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a> Dedicated SRE teams are common at larger <a href="/facts/Web_development/UW5ypSzj">web development</a> companies.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a> In middle-sized and smaller companies, DevOps teams sometimes perform SRE, as well.<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a> Organizations that have adopted the concept include <a href="/facts/Airbnb/EP95k6Rr">Airbnb</a>, <a href="/facts/Dropbox_(service)/1WHn5C7p">Dropbox</a>, <a href="/facts/IBM/QszA7nwd">IBM</a>,<a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a> <a href="/facts/LinkedIn/Iq2IcNMt">LinkedIn</a>,<a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a> <a href="/facts/Netflix/TKtlFVY9">Netflix</a>,<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a> and <a href="/facts/Wikimedia_Foundation/W1XHdpK2">Wikimedia</a>.<a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a> </p> <h2 id="definition">Definition</h2> <p>Site reliability engineers (SREs) are responsible for a combination of system <a href="/facts/Availability/l8SGSyyf">availability</a>, <a href="/facts/Latency_(engineering)/6Bjhn30H">latency</a>, <a href="/facts/Computer_performance/td1qKox4">performance</a>, efficiency, <a href="/facts/Change_management_(engineering)/of63a9WE">change management</a>, <a href="/facts/Website_monitoring/cpYcgR4L">monitoring</a>, <a href="/facts/Incident_management/KPwTbXbp">emergency response</a>, and <a href="/facts/Capacity_planning/VqbuA3C3">capacity planning</a>.<a class="footnote-ref" id="fnref:14" href="#fn:14"><sup>14</sup></a> SREs often have backgrounds in <a href="/facts/Software_engineering/KX6cE9kz">software engineering</a>, <a href="/facts/Systems_engineering/gFwRmkDp">systems engineering</a>, and/or <a href="/facts/System_administrator/zNhThrh9">system administration</a>.<a class="footnote-ref" id="fnref:15" href="#fn:15"><sup>15</sup></a> The focuses of SRE include <a href="/facts/Automation/3YlF2K5q">automation</a>, <a href="/facts/Systems_design/hFWYoaNP">system design</a>, and improvements to <a href="/facts/Robustness_(computer_science)/jHr1dT3m">system resilience</a>.<a class="footnote-ref" id="fnref:16" href="#fn:16"><sup>16</sup></a> </p><p>SRE is considered a specific implementation of DevOps;<a class="footnote-ref" id="fnref:17" href="#fn:17"><sup>17</sup></a> focusing specifically on building reliable systems, whereas DevOps covers a broader scope of operations.<a class="footnote-ref" id="fnref:18" href="#fn:18"><sup>18</sup></a><a class="footnote-ref" id="fnref:19" href="#fn:19"><sup>19</sup></a><a class="footnote-ref" id="fnref:20" href="#fn:20"><sup>20</sup></a> Despite having different focuses, some companies have rebranded their operations teams to SRE teams.<a class="footnote-ref" id="fnref:21" href="#fn:21"><sup>21</sup></a> </p> <h2 id="principles-and-practices">Principles and practices</h2> <p>Common definitions of the practices include (but are not limited to):<a class="footnote-ref" id="fnref:22" href="#fn:22"><sup>22</sup></a><a class="footnote-ref" id="fnref:23" href="#fn:23"><sup>23</sup></a> </p> <ul><li><a href="/facts/Automation_and_Remote_Control/TadtuQt2">Automation</a> of repetitive tasks for cost-effectiveness.</li> <li>Defining reliability goals to prevent endless effort.</li> <li>Design of systems with a goal to reduce risks to availability, latency, and efficiency.</li> <li><a href="/facts/Software_observability/q5GGWYH7">Observability</a>, the ability to ask arbitrary questions about a system without having to know ahead of time what to ask.<a class="footnote-ref" id="fnref:24" href="#fn:24"><sup>24</sup></a></li></ul> <p>Common definitions of the principles include (but are not limited to): </p> <ul><li><a href="https://sre.google/sre-book/eliminating-toil/">Toil management</a>, the implementation of the first principle outlined above.</li> <li>Defining and measuring reliability goals—<a href="/facts/Service_level_indicator/GXF5wBxw">SLIs</a>, <a href="/facts/Service-level_objective/nI1N2j67">SLOs</a>, and error budgets.</li> <li>Non-Abstract Large Scale Systems Design (<a href="https://sre.google/workbook/non-abstract-design/">NALSD</a>) with a focus on reliability.</li> <li>Designing for and implementing observability.</li> <li>Defining, testing, and running an <a href="/facts/Incident_management/KPwTbXbp">incident management</a> process.</li> <li><a href="/facts/Capacity_planning/VqbuA3C3">Capacity planning</a>.</li> <li>Change and release management, including <a href="/facts/CI%2fCD/uGeanHaX">CI/CD</a>.</li> <li><a href="/facts/Chaos_engineering/7A6L9qoA">Chaos engineering</a>.</li></ul> <h2 id="deployment">Deployment</h2> <p>SRE teams collaborate with other departments within organizations to guide the implementation of the mentioned principles. Below is an overview of common practices:<a class="footnote-ref" id="fnref:25" href="#fn:25"><sup>25</sup></a> </p> <h3>Kitchen Sink</h3> <p><a href="https://cloud.google.com/blog/products/devops-sre/how-sre-teams-are-organized-and-how-to-get-started">Kitchen Sink</a> refers to the expansive and often unbounded scope of services and workflows that SRE teams oversee. Unlike traditional roles with clearly defined boundaries, SREs are tasked with various responsibilities, including system performance optimization, incident management, and automation. This approach allows SREs to address multiple challenges, ensuring that systems run efficiently and evolve in response to changing demands and complexities. </p> <h3>Infrastructure</h3> <p>Infrastructure SRE teams focus on maintaining and improving the reliability of systems that support other teams' workflows. While they sometimes collaborate with platform engineering teams, their primary responsibility is ensuring up-time, performance, and efficiency. Platform teams, on the other hand, primarily develop the software and systems used across the organization. While reliability is a goal for both, platform teams prioritize creating and maintaining the tools and services used by internal stakeholders, whereas Infrastructure SRE teams are tasked with ensuring those systems run smoothly and meet reliability standards. </p> <h3>Tools</h3> <p>SRE teams utilize a variety of tools with the aim of measuring, maintaining, and enhancing system reliability. These tools play a role in monitoring performance, identifying issues, and facilitating proactive maintenance. For instance, <a href="/facts/Nagios_Core/y3E81GSW">Nagios Core</a> is commonly employed for system monitoring and alerting, while <a href="/facts/Prometheus_(software)/Vlcfymdn">Prometheus (software)</a> is frequently used for collecting and querying metrics in cloud-native environments. </p> <h3>Product or Application</h3> <p>SRE teams dedicated to specific products or applications are common in large organizations.<a class="footnote-ref" id="fnref:26" href="#fn:26"><sup>26</sup></a> These teams are responsible for ensuring the reliability, scalability, and performance of key services. In larger companies, it's typical to have multiple SRE teams, each focusing on different products or applications, ensuring that each area receives specialized attention to meet performance and availability targets. </p> <h3>Embedded</h3> <p>In an embedded model, individual SREs or small SRE pairs are integrated within software engineering teams. These SREs collaborate with developers, applying core SRE principles—such as automation, monitoring, and incident response—directly to the software development lifecycle. This approach aims to enhance reliability, performance, and collaboration between SREs and developers. </p> <h3>Consulting</h3> <p>Consulting SRE teams specialize in advising organizations on the implementation of SRE principles and practices. Typically composed of seasoned SREs with a history across various implementations, these teams provide insights and guidance for specific organizational needs. When working directly with clients, these SREs are often referred to as '<a href="https://cloud.google.com/blog/products/devops-sre/introducing-a-new-era-of-customer-support-google-customer-reliability-engineering">Customer Reliability Engineers</a>.' </p><p>In large organizations that have adopted SRE, a hybrid model is common. This model includes various implementations, such as multiple Product/Application SRE teams dedicated to addressing the specific reliability needs of different products. An Infrastructure SRE team may collaborate with a Platform engineering group to achieve shared reliability goals for a unified platform that supports all products and applications. </p> <h2 id="industry">Industry</h2> <p>Since 2014, the <a href="/facts/USENIX/2wz6aQwq">USENIX</a> organization has hosted the annual <a href="https://www.usenix.org/srecon">SREcon</a> conference, bringing together site reliability engineers from various industries. This conference is a platform for professionals to share knowledge, explore effective practices, and discuss trends in site reliability engineering.<a class="footnote-ref" id="fnref:27" href="#fn:27"><sup>27</sup></a> </p> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Chaos_engineering/7A6L9qoA">Chaos engineering</a></li> <li><a href="/facts/Cloud_computing/guhg0FGV">Cloud computing</a></li> <li><a href="/facts/Data_center/DS5wJY41">Data center</a></li> <li><a href="/facts/IT_disaster_recovery/gDARyxfl">Disaster recovery</a></li> <li><a href="/facts/High_availability_software/HbSU3tJ0">High availability software</a></li> <li><a href="/facts/Infrastructure_as_code/QDx3sDck">Infrastructure as code</a></li> <li><a href="/facts/Operations%2c_administration_and_management/3nsCyyeB">Operations, administration and management</a></li> <li><a href="/facts/Operations_management/jTNFINl1">Operations management</a></li> <li><a href="/facts/Reliability_engineering/cd6AS5w3">Reliability engineering</a></li> <li><a href="/facts/System_administration/zNhThrh9">System administration</a></li> <li><a href="/facts/Backup_site/4tgNNUXm">Backup site</a></li></ul> <h2 id="further-reading">Further reading</h2> <ul><li>Limoncelli, Tom; Chalup, Strata R.; Hogan, Christina J. (September 2014). <a href="https://www.worldcat.org/oclc/891786231"><i>The Practice of Cloud System Administration: DevOps and SRE Practices for Web Services</i></a>. Vol. 2. Upper Saddle River, NJ: <a href="/facts/Addison-Wesley/A7UFATEG">Addison-Wesley</a>. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-0133478549. <a href="/facts/OCLC_(identifier)/Yqad8waq">OCLC</a> <a href="https://search.worldcat.org/oclc/891786231">891786231</a>.</li> <li>Beyer, Betsy; Jones, Chris; Petoff, Jennifer; Murphy, Niall Richard, eds. (2016). <a href="https://archive.org/details/sitereliabilitye0000unse"><i>Site Reliability Engineering: How Google Runs Production Systems</i></a>. <a href="/facts/O%2527Reilly_Media/Fr3Kv7sm">O'Reilly</a>. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-1491929124.</li> <li>Blank-Edelman, David N., ed. (2018). <a href="https://www.worldcat.org/oclc/1052565720"><i>Seeking SRE: Conversations About Running Production Systems at Scale</i></a> (1 ed.). Sebastopol, CA: O'Reilly. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-1491978863. <a href="/facts/OCLC_(identifier)/Yqad8waq">OCLC</a> <a href="https://search.worldcat.org/oclc/1052565720">1052565720</a>.</li> <li>Beyer, Betsy; Murphy, Niall; Kawahara, Kent; Rensin, David; Thorne, Stephen (2018). <i>The Site Reliability Workbook: Practical Ways to Implement SRE</i>. O'Reilly. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-1492029502.</li> <li>Welch, Nat (2018). <i>Real-World SRE: The Survival Guide for Responding to a System Outage and Maximizing Uptime</i>. <a href="/facts/Packt/eT0RTSSK">Packt</a>. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-1788628884.</li> <li>Adkins, Heather; Beyer, Betsy; Blankinship, Paul; Lewandowski, Piotr; Oprea, Ana; Stubblefield, Adam (2020). <i>Building Secure and Reliable Systems: Best Practices for Designing, Implementing, and Maintaining Systems</i>. O'Reilly. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-1-4920-8312-2. <a href="/facts/OCLC_(identifier)/Yqad8waq">OCLC</a> <a href="https://search.worldcat.org/oclc/1129470292">1129470292</a>.</li> <li>Rosenthal, Jones, Casey, Nora (2020). <i>Chaos Engineering: System Resiliency in Practice</i>. O'Reilly. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-1492043867.{{cite book}}: CS1 maint: multiple names: authors list (link)</li></ul> <h2 id="external-links">External links</h2> <ul><li><a href="https://github.com/dastergon/awesome-sre">Awesome Site Reliability Engineering</a> resources list</li> <li><a href="https://github.com/upgundecha/howtheysre">How they SRE</a> resources list</li> <li><a href="https://sreweekly.com/">SRE Weekly</a> weekly newsletter devoted to SRE</li> <li><a href="https://sre.google/">SRE at Google</a> landing page for learning more about SRE in Google</li> <li><a href="https://komodor.com/learning-center/?category=troubleshooting/">Komodor K8s Reliability</a> learning centre with resources for SREs working with Kubernetes</li> <li><a href="https://diamantinoalmeida.com/sre-what-do-you-need-to-know-to-master-this-role/">SRE: What Do You Need To Know To Master This Role?</a> resource list</li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>"What is SRE? - Site Reliability Engineering Explained - AWS". Amazon Web Services, Inc. Retrieved 2024-12-26. <a href="https://aws.amazon.com/what-is/sre/" target="_blank">https://aws.amazon.com/what-is/sre/</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>"Evaluating where your team lies on the SRE spectrum". Google Cloud Blog. Retrieved 2021-06-26. <a href="https://cloud.google.com/blog/products/devops-sre/evaluating-where-your-team-lies-on-the-sre-spectrum/" target="_blank">https://cloud.google.com/blog/products/devops-sre/evaluating-where-your-team-lies-on-the-sre-spectrum/</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Hill, Patrick. "Love DevOps? Wait until you meet SRE". Atlassian. Retrieved June 17, 2021. <a href="https://www.atlassian.com/incident-management/devops/sre" target="_blank">https://www.atlassian.com/incident-management/devops/sre</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>"What is SRE?". Red Hat. Retrieved June 17, 2021. <a href="https://www.redhat.com/en/topics/devops/what-is-sre" target="_blank">https://www.redhat.com/en/topics/devops/what-is-sre</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>Treynor, Ben (2014). "Keys to SRE". USENIX SREcon14. Retrieved June 17, 2021. <a href="https://www.usenix.org/conference/srecon14/technical-sessions/presentation/keys-sre" target="_blank">https://www.usenix.org/conference/srecon14/technical-sessions/presentation/keys-sre</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>Gossett, Stephen (June 1, 2020). "What Is a Site Reliability Engineer? What Does an SRE Do?". Built In. Retrieved June 17, 2021. <a href="https://builtin.com/software-engineering-perspectives/site-reliability-engineer" target="_blank">https://builtin.com/software-engineering-perspectives/site-reliability-engineer</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>Fischer, Donald (March 2, 2016). "Are site reliability engineers the next data scientists?". TechCrunch. Archived from the original on August 12, 2019. Retrieved June 17, 2021. <a href="https://techcrunch.com/2016/03/02/are-site-reliability-engineers-the-next-data-scientists/" target="_blank">https://techcrunch.com/2016/03/02/are-site-reliability-engineers-the-next-data-scientists/</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>Beres, Cristi (October 9, 2024). "SRE & DevOps: Striking the Perfect IT Match". Synergo Group.{{cite web}}: CS1 maint: url-status (link) <a href="https://synergogroup.net/sre-devops/" target="_blank">https://synergogroup.net/sre-devops/</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>Gossett, Stephen (June 1, 2020). "What Is a Site Reliability Engineer? What Does an SRE Do?". Built In. Retrieved June 17, 2021. <a href="https://builtin.com/software-engineering-perspectives/site-reliability-engineer" target="_blank">https://builtin.com/software-engineering-perspectives/site-reliability-engineer</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>"Site Reliability Engineering". IBM Cloud Education. IBM. November 12, 2020. Retrieved June 21, 2021. <a href="https://www.ibm.com/cloud/learn/site-reliability-engineering" target="_blank">https://www.ibm.com/cloud/learn/site-reliability-engineering</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>"Site Reliability Engineering (SRE)". engineering.linkedin.com. Retrieved March 12, 2024. <a href="https://engineering.linkedin.com/india-engineering/site-reliability-engineers" target="_blank">https://engineering.linkedin.com/india-engineering/site-reliability-engineers</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>Fischer, Donald (March 2, 2016). "Are site reliability engineers the next data scientists?". TechCrunch. Archived from the original on August 12, 2019. Retrieved June 17, 2021. <a href="https://techcrunch.com/2016/03/02/are-site-reliability-engineers-the-next-data-scientists/" target="_blank">https://techcrunch.com/2016/03/02/are-site-reliability-engineers-the-next-data-scientists/</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>"SRE - Wikitech". wikitech.wikimedia.org. Retrieved 2021-10-17. <a href="https://wikitech.wikimedia.org/wiki/SRE" target="_blank">https://wikitech.wikimedia.org/wiki/SRE</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> <li id="fn:14"><p>Treynor, Ben. "In Conversation" (Interview). Interviewed by Niall Murphy. Google Site Reliability Engineering. <a href="https://sre.google/in-conversation/" target="_blank">https://sre.google/in-conversation/</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li> <li id="fn:15"><p>Jones, Chris; Underwood, Todd; Nukala, Shylaja (June 2015). "Hiring Site Reliability Engineers" (PDF). ;login:. Vol. 40, no. 3. pp. 35–39. Archived (PDF) from the original on August 24, 2017. Retrieved June 17, 2021. <a href="https://www.usenix.org/system/files/login/articles/login_june_07_jones.pdf" target="_blank">https://www.usenix.org/system/files/login/articles/login_june_07_jones.pdf</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></p></li> <li id="fn:16"><p>Jones, Chris; Underwood, Todd; Nukala, Shylaja (June 2015). "Hiring Site Reliability Engineers" (PDF). ;login:. Vol. 40, no. 3. pp. 35–39. Archived (PDF) from the original on August 24, 2017. Retrieved June 17, 2021. <a href="https://www.usenix.org/system/files/login/articles/login_june_07_jones.pdf" target="_blank">https://www.usenix.org/system/files/login/articles/login_june_07_jones.pdf</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></p></li> <li id="fn:17"><p>Dave Harrison (9 Oct 2018). "Interview with Betsy Beyer, Stephen Thorne of Google". Retrieved 24 July 2024. <a href="https://driftboatdave.com/2018/10/09/interview-with-betsy-beyer-stephen-thorne-of-google/" target="_blank">https://driftboatdave.com/2018/10/09/interview-with-betsy-beyer-stephen-thorne-of-google/</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></p></li> <li id="fn:18"><p>Beyer, Betsy; Jones, Chris; Petoff, Jennifer; Murphy, Niall, eds. (2016). Site Reliability Engineering: How Google Runs Production Systems. Sebastopol, CA: O'Reilly Media. ISBN 978-1-4919-5118-7. OCLC 945577030. <a href="978-1-4919-5118-7" target="_blank">978-1-4919-5118-7</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></p></li> <li id="fn:19"><p>Vargo, Seth; Fong-Jones, Liz (March 1, 2018). What's the Difference Between DevOps and SRE? (class SRE implements DevOps) (Video). Google. Archived from the original on December 3, 2019. Retrieved March 8, 2018. <a href="/wiki/Liz_Fong-Jones" target="_blank">/wiki/Liz_Fong-Jones</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></p></li> <li id="fn:20"><p>"What is SRE? - SRE Explained - AWS". Amazon Web Services, Inc. Retrieved 2022-11-05. <a href="https://aws.amazon.com/what-is/sre/" target="_blank">https://aws.amazon.com/what-is/sre/</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></p></li> <li id="fn:21"><p>Gossett, Stephen (June 1, 2020). "What Is a Site Reliability Engineer? What Does an SRE Do?". Built In. Retrieved June 17, 2021. <a href="https://builtin.com/software-engineering-perspectives/site-reliability-engineer" target="_blank">https://builtin.com/software-engineering-perspectives/site-reliability-engineer</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></p></li> <li id="fn:22"><p>"Evaluating where your team lies on the SRE spectrum". Google Cloud Blog. Retrieved 2021-06-26. <a href="https://cloud.google.com/blog/products/devops-sre/evaluating-where-your-team-lies-on-the-sre-spectrum/" target="_blank">https://cloud.google.com/blog/products/devops-sre/evaluating-where-your-team-lies-on-the-sre-spectrum/</a> <a href="#fnref:22" class="footnote-back-ref">↩</a></p></li> <li id="fn:23"><p>"The 7 SRE Principles [And How to Put Them Into Practice]". www.blameless.com. Retrieved 2021-06-26. <a href="https://www.blameless.com//blog/sre-principles" target="_blank">https://www.blameless.com//blog/sre-principles</a> <a href="#fnref:23" class="footnote-back-ref">↩</a></p></li> <li id="fn:24"><p>"Learn about observability | Honeycomb". docs.honeycomb.io. Retrieved 2021-06-26. <a href="https://docs.honeycomb.io/getting-started/learning-about-observability/" target="_blank">https://docs.honeycomb.io/getting-started/learning-about-observability/</a> <a href="#fnref:24" class="footnote-back-ref">↩</a></p></li> <li id="fn:25"><p>"SRE at Google: How to structure your SRE team". Google Cloud Blog. Retrieved 2021-06-26. <a href="https://cloud.google.com/blog/products/devops-sre/how-sre-teams-are-organized-and-how-to-get-started/" target="_blank">https://cloud.google.com/blog/products/devops-sre/how-sre-teams-are-organized-and-how-to-get-started/</a> <a href="#fnref:25" class="footnote-back-ref">↩</a></p></li> <li id="fn:26"><p>"SRE at Google: How to structure your SRE team". Google Cloud Blog. Retrieved 2024-11-11. <a href="https://cloud.google.com/blog/products/devops-sre/how-sre-teams-are-organized-and-how-to-get-started" target="_blank">https://cloud.google.com/blog/products/devops-sre/how-sre-teams-are-organized-and-how-to-get-started</a> <a href="#fnref:26" class="footnote-back-ref">↩</a></p></li> <li id="fn:27"><p>"Usenix SREcon". USENIX. 2021. Retrieved June 17, 2021. <a href="https://www.usenix.org/srecon" target="_blank">https://www.usenix.org/srecon</a> <a href="#fnref:27" class="footnote-back-ref">↩</a></p></li> </ol>