{"id":269429,"date":"2020-10-05T11:32:32","date_gmt":"2020-10-05T16:32:32","guid":{"rendered":"https:\/\/www.webscale.com\/?p=269293"},"modified":"2023-12-29T08:05:53","modified_gmt":"2023-12-29T13:05:53","slug":"without-observability-devops-is-doomed","status":"publish","type":"post","link":"https:\/\/www.webscale.com\/blog\/without-observability-devops-is-doomed\/","title":{"rendered":"Without Observability, DevOps is Doomed"},"content":{"rendered":"

Speed of innovation is a key differentiator for businesses in a competitive market. Developers working at the bleeding edge of technology are always looking for ways to iterate quickly, yet safely. Observability helps DevOps teams move quickly with greater confidence.<\/p>\n

As Arijit Mukherji laid out in his opening talk at a\u00a0recent Splunk event<\/a>, DevOps and observability go hand-in-hand.<\/p>\n

\u201cMove fast and break things. If we don\u2019t move fast, the competition will beat us. That\u2019s the reality. 52% of Fortune 500 companies have disappeared since 2000. Without DevOps services, digital transformation is doomed because it won\u2019t be able to keep up.\u00a0Without observability, DevOps is doomed.<\/strong>\u00a0Observability provides me with vision. Imagine DevOps as a fast car, but to stop it from going in circles or crashing headlong into a wall, I need to know where I\u2019m going. That\u2019s the lens observability provides.\u201d– Arijit Mukherji, Distinguished Architect, Splunk (formerly CTO at SignalFx)<\/cite><\/p><\/blockquote>\n

Are There Three Pillars of Observability?<\/h3>\n

Kaslin Fields, Developer Advocate at Google (who illustrates cloud-native concepts as comic books<\/a>) describes observability as \u201cthe board you see at an amusement park that tells you the wait time and whether or not a ride is closed\u201d, i.e. knowing your systems so you know what\u2019s going on. Robust observability tooling can help you iterate more quickly by providing visibility into every stage of the development lifecycle.<\/p>\n

The three commonly defined pillars of observability are logs, metrics, and tracing. (Thanks to\u00a0O\u2019Reilly<\/a>\u00a0for the definitions.)<\/p>\n

Logs<\/h4>\n

\u201can immutable, timestamped record of discrete events that happened over time\u201d<\/strong><\/em><\/p>\n

Logs let you see who the last person in the system was before you and help surface highly granular local information. Since a log is simply a string or a blob of JSON or typed key-value pairs, any data can be represented in the form of a log line.<\/p>\n

There are three forms of event logs:<\/p>\n

    \n
  1. plaintext – the most common format of logs; might be free-form text;<\/li>\n
  2. structured – becoming more popular, typically in JSON format;<\/li>\n
  3. binary – in the Protobuf format, MySQL binlogs, or the pflog format.<\/li>\n<\/ol>\n

    Metrics<\/h4>\n

    \u201ca numeric representation of data measured over intervals of time\u201d<\/strong><\/em><\/p>\n

    Metrics allow you to track trends and patterns in the behavior of a system over set intervals of time, making them ideally suited to building dashboards that reflect historical trends.<\/p>\n

    Tracing<\/h4>\n

    \u201ca trace is a representation of causally related distributed events that encode the end-to-end request flow through a distributed system\u201d<\/strong><\/em><\/p>\n

    Traces are a representation of logs. They offer visibility into the path traveled by a request and its structure. Traces help system admins understand the amount of work performed at each layer and preserve causality by using happens-before semantics.<\/p>\n

    Despite widespread adoption of these definitions, there is a counter argument out there that these three pillars are too narrow and there should be a wider definition of observability tooling.<\/p>\n

    \u201cYou may achieve observability with all three, or none [of the pillars] – what matters is what you do with the data, not the data itself.\u201dBen Sigelman, CEO & Co-Founder, LightStep<\/a><\/cite><\/p><\/blockquote>\n

    His mindset is that these pillars are most appropriate for enterprises of \u201cplanet-scale\u201d, such as Google, Facebook or Twitter, and engineers at smaller organizations need to find \u201ca new scorecard for observability.<\/a>\u201d Charity Majors, CEO of Honeycomb, instead of listing preset tools defines\u00a0a set of technical specifications<\/a>\u00a0that observability tooling should have. The debate continues.<\/p>\n

    The Growing Popularity of Observability<\/h3>\n

    \u201cIn 2020, observability is making the transition from being a niche concern to becoming a new frontier for user experience, systems and service management in web companies and enterprises alike.\u201d \u2013\u00a0James Governor, RedMonk<\/a><\/p><\/blockquote>\n

    There are various key drivers of the rise in observability interest and adoption over the last few years. These include a rise in uncertainty in systems and apps and an increase in complexity tied to the rise of microservices and containerization. Visibility into these complex ecosystems is essential.<\/p>\n

    Feature management and experimentation require the ability to query the system from end to end at any given time in order to understand performance levels, detect bugs and execute fixes. In addition, as mentioned, there is an increased demand for regular changes and updates. Indeed, change is the goal in today\u2019s development environment and observability provides the visibility necessary to give developers confidence to move quickly.<\/p>\n

    There is often a debate over whether observability is only necessary when companies reach a certain scale. However,\u00a0in Charity Majors\u2019 words<\/a>,<\/p>\n

    \u201cDeveloping software with observability is better at ANY scale. It\u2019s better for monoliths, it\u2019s better for tiny one-person teams, it\u2019s better for pre-production services, it\u2019s better for literally everyone always. The sooner and earlier you adopt it, the more compounding value you will reap over time, and the more of your engineers\u2019 time will be devoted to forward progress and creating value.\u201dCharity Majors, CEO, Honeycomb<\/cite><\/p><\/blockquote>\n

    The Role of Observability at Every Stage of the Development Lifecycle<\/h3>\n

    There are several critical initiatives where observability should play a particularly central role. These include:<\/p>\n

      \n
    • Hybrid\/multi-cloud monitoring<\/li>\n
    • Cloud migration<\/li>\n
    • Application monitoring<\/li>\n
    • Incident response<\/li>\n<\/ul>\n

      However, it\u2019s clear that observability plays a critical role in shining a light on software systems at every stage of the development lifecycle. It provides an essential feedback loop for Engineering, Ops, SREs and DevOps teams to improve end user experience.<\/p>\n

      Ultimately, observability is about understanding performance from the end user point of view. How well does your request execute from end to end? It\u2019s essential to understand the entire service delivery chain. Observability provides that lens from the perspective of the user.<\/p>\n

      Finally, observability is a mindset and set of standards as much as it is a technology. The tools you adopt and how you use it will, of course, depend on your specific needs.<\/p>\n","protected":false},"excerpt":{"rendered":"

      Speed of innovation is a key differentiator for businesses in a competitive market. Developers working at the bleeding edge of technology are always looking for ways to iterate quickly, yet safely. Observability helps DevOps teams move quickly with greater confidence. As Arijit Mukherji laid out in his opening talk at a\u00a0recent Splunk event, DevOps and […]<\/p>\n","protected":false},"author":36,"featured_media":269448,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","rank_math_lock_modified_date":false,"_aioseo_description":"","_aioseo_keywords":"","_aioseo_og_article_section":"","_aioseo_og_article_tags":"","_aioseo_og_description":"","_aioseo_og_title":"","_aioseo_title":"","_aioseo_twitter_description":"","_aioseo_twitter_title":"","_author_photo":"field_6513304084a08","_doc_url":"","_dp_original":"269290","_et_autogenerated_title":"","_et_body_layout_enabled":"","_et_body_layout_id":"","_et_builder_dynamic_assets_loading_attr_threshold":"2","_et_builder_module_features_cache":null,"_et_builder_version":"","_et_default":"","_et_enabled":"","_et_footer_layout_enabled":"","_et_footer_layout_id":"","_et_header_layout_enabled":"","_et_header_layout_id":"","_et_pb_ab_current_shortcode":"","_et_pb_ab_subjects":"","_et_pb_built_for_post_type":"","_et_pb_custom_css":"","_et_pb_enable_shortcode_tracking":"","_et_pb_excluded_global_options":"","_et_pb_first_image":"","_et_pb_gutter_width":"","_et_pb_module_type":"","_et_pb_page_layout":"et_right_sidebar","_et_pb_page_z_index":"","_et_pb_post_hide_nav":"default","_et_pb_row_layout":"","_et_pb_show_page_creation":"","_et_pb_show_title":"on","_et_pb_side_nav":"off","_et_pb_static_css_file":"","_et_pb_truncate_post":"","_et_pb_truncate_post_date":"","_et_post_bg_color":"#ffffff","_et_post_bg_layout":"light","_et_template":[],"_et_theme_builder_marked_as_unused":"","_et_use_on":"","_gallery_link_target":"","_global_colors_info":"","_lh_copy_from_url-original_file":"","_version_history":"","_wp_old_date":["2023-11-09"],"_wpcode_auto_insert":"","_wpcode_auto_insert_number":"","_wpcode_conditional_logic":[],"_wpcode_conditional_logic_enabled":"","_wpcode_library_id":"","_wpcode_library_version":"","_wpcode_location_extra":"","_wpcode_note":"","_wpcode_priority":"","_wpcode_shortcode_attributes":[],"_wpmf_gallery_custom_image_link":"","ao_post_optimize":[],"author_photo":"268246","doc_url":"","et_enqueued_post_fonts":{"family":{"et-gf-lato":"Lato:100,100italic,300,300italic,regular,italic,700,700italic,900,900italic"},"subset":["latin","latin-ext"],"cache_key":"{\"gph\":0,\"divi\":\"4.24.1\",\"wp\":\"6.6.2\",\"enable_all_character_sets\":\"false\"}"},"rank_math_contentai_score":{"wordCount":"100","linkCount":"0","headingCount":"100","mediaCount":"62.22"},"rank_math_description":"","rank_math_facebook_image":"","rank_math_facebook_image_id":"","rank_math_internal_links_processed":["1","1","1","1","1","1","1"],"rank_math_og_content_image":[],"rank_math_seo_score":"20","rank_math_title":"","version_history":"","wp-smpro-smush-data":[],"wp-smush-animated":"","wpmf_filetype":"","wpmf_order":"","wpmf_size":"","_":"","_bj_lazy_load_skip_post":[],"_divi_filters_post_type":"","_et_dynamic_cached_attributes":{"sticky_position":["top"],"use_custom_gutter":["on"],"fullwidth":["off"],"button_icon":["$||divi||400"],"social_network":["facebook","twitter","linkedin","youtube","last_fm"],"header_2_font":"|800|||||||","header_2_font_size":"34px","header_2_text_align":"center","animation_intensity_slide":"10%","animation_duration":"800ms","animation_delay":"15ms","animation_intensity_zoom":"15%","animation_intensity_flip":"15%","animation_intensity_fold":"15%","animation_intensity_roll":"15%","animation_direction":"center","animation_style":"none","background_color_gradient_start":"rgba(255,255,255,0)","background_color_gradient_end":"#fafafa","custom_padding":"120px||0px||false|false","background_color_gradient_stops":"rgba(255,255,255,0) 0%|#fafafa 100%","custom_padding_last_edited":"on|desktop","custom_padding_tablet":"||30px||false|false","custom_padding_phone":"60px||||false|false"},"_et_dynamic_cached_shortcodes":["et_pb_post_content","et_pb_contact_field","et_pb_signup_custom_field","et_pb_social_media_follow_network","et_pb_section","et_pb_row","et_pb_column","et_pb_blog","et_pb_blurb","et_pb_button","et_pb_code","et_pb_contact_form","et_pb_post_nav","et_pb_post_title","et_pb_signup","et_pb_social_media_follow","et_pb_text"],"_et_pb_ab_bounce_rate_limit":"","_et_pb_ab_stats_refresh_interval":[],"_et_pb_content_area_background_color":"","_et_pb_dark_text_color":"","_et_pb_light_text_color":"","_et_pb_section_background_color":"","_job_location":"","_job_locations":"","_links_to":"","_links_to_target":"","_product_image_gallery":"","_schema_code":"","_synced_version":"","_wp_attachment_context":"","_wp_attachment_image_alt":[],"_wpie_source_url":"","_yoast_wpseo_content_score":"","_yoast_wpseo_focuskeywords":"","_yoast_wpseo_metadesc":"","_yoast_wpseo_opengraph-image":"","_yst_prominent_words_version":"","inline_featured_image":[],"job_location":[],"job_locations":"","options":"","original-file":"","post_views_count":"","rank_math_analytic_object_id":"","rank_math_canonical_url":"","rank_math_focus_keyword":[],"rank_math_news_sitemap_robots":"","rank_math_primary_category":"111","rank_math_primary_ccategory":"","rank_math_primary_job_locations":"","rank_math_primary_partners_category":"","rank_math_primary_pr_category":"","rank_math_primary_press_release_year":"","rank_math_rich_snippet":"","rank_math_robots":[],"rank_math_schema_Article":[],"rank_math_schema_Organization":[],"rank_math_schema_VideoObject":[],"rank_math_shortcode_schema_s-23675683-fff5-4300-88fe-da8afc8b1bb9":"","rank_math_shortcode_schema_s-307bbc91-c6b1-41aa-950d-c50d435a949c":"","rank_math_shortcode_schema_s-63a052dbc0384":"","rank_math_shortcode_schema_s-63a052dbc039d":"","rank_math_shortcode_schema_s-63a052dbc03a6":"","rank_math_shortcode_schema_s-63a052dbc03aa":"","rank_math_shortcode_schema_s-63a052dbc03b5":"","rank_math_shortcode_schema_s-63a052dbc03ba":"","rank_math_shortcode_schema_s-63a052dbc03bd":"","rank_math_shortcode_schema_s-63b6dd7d53a96":"","rank_math_shortcode_schema_s-63b6dd7d53a9f":"","rank_math_shortcode_schema_s-63b6dd7d53aa2":"","rank_math_shortcode_schema_s-63b6dd7d53aa4":"","rank_math_shortcode_schema_s-63b6dd7d53aa7":"","rank_math_shortcode_schema_s-63b6dd7d53aa9":"","rank_math_shortcode_schema_s-63b6dd7d53aab":"","rank_math_shortcode_schema_s-63b6dd7d53aad":"","rank_math_shortcode_schema_s-63b6dd7d53aaf":"","rank_math_shortcode_schema_s-63c15fcf43311":"","rank_math_shortcode_schema_s-63c15fcf43322":"","rank_math_shortcode_schema_s-63c15fcf43325":"","rank_math_shortcode_schema_s-63c15fcf43327":"","rank_math_shortcode_schema_s-63c15fcf43329":"","rank_math_shortcode_schema_s-63c15fcf4332a":"","rank_math_shortcode_schema_s-63c15fcf4332c":"","rank_math_shortcode_schema_s-63c15fcf4332e":"","rank_math_shortcode_schema_s-63c15fcf43330":"","rank_math_shortcode_schema_s-63f52c5ed40bb":"","rank_math_shortcode_schema_s-6409f40a9b7d5":"","rank_math_shortcode_schema_s-64354a3892419":"","rank_math_shortcode_schema_s-6440158136148":"","rank_math_shortcode_schema_s-6446d2f9353ee":"","rank_math_shortcode_schema_s-6446d2f9353f3":"","rank_math_shortcode_schema_s-6447c0fe4673c":"","rank_math_shortcode_schema_s-64e4d743542d7":"","schema_code":"","smush-complete":"","smush-info":"","smush-stats":[],"synced_version":"","wpmf_remote_video_link":"","_exp":"","_inc":"","_mc4wp_settings":[],"_post-subtitle":"","_pwh_dcfh_contact_email":"","_pwh_dcfh_contact_form_id":"","_pwh_dcfh_form_fields":"","_pwh_dcfh_ip_address":"","_pwh_dcfh_page_id":"","_pwh_dcfh_read_by":"","_pwh_dcfh_referer_url":"","_pwh_dcfh_user_agent":[],"_section1_col1":"","_section1_col2":"","_section1_col3":"","_section1_col4":"","_section2_col1":"","_section2_col2":"","_section2_col3":"","_section2_col4":"","_section2_col5":"","_section2_col6":"","_section3_col1":"","_section3_col2":"","_section3_col3":"","_section3_col4":"","_section3_col5":"","_section3_col6":"","_section4_col1":"","_section4_col2":"","_section4_col3":"","_section4_col4":"","_section4_col5":"","_section4_col6":"","_section5_col1":"","_section5_col2":"","_section5_col3":"","_section5_col4":"","_section5_col5":"","_section5_col6":"","_section6_col1":"","_section6_col2":"","_section6_col3":"","_section6_col4":"","_section6_col5":"","_section6_col6":"","_select_author":"","_test":"","_wp_attachment_backup_sizes":[],"_yoast_wpseo_estimated-reading-time-minutes":[],"_yoast_wpseo_focuskw":[],"_yoast_wpseo_focuskw_text_input":[],"_yoast_wpseo_linkdex":[],"_yoast_wpseo_meta-robots-nofollow":[],"_yoast_wpseo_meta-robots-noindex":[],"_yoast_wpseo_primary_category":[],"_yoast_wpseo_title":[],"_yoast_wpseo_wordproof_timestamp":"","exp":"","inc":"","post-subtitle":[],"rank_math_schema_BlogPosting":[],"section1_col1":"","section1_col2":"","section1_col3":"","section1_col4":"","section2_col1":"","section2_col2":"","section2_col3":"","section2_col4":"","section2_col5":"","section2_col6":"","section3_col1":"","section3_col2":"","section3_col3":"","section3_col4":"","section3_col5":"","section3_col6":"","section4_col1":"","section4_col2":"","section4_col3":"","section4_col4":"","section4_col5":"","section4_col6":"","section5_col1":"","section5_col2":"","section5_col3":"","section5_col4":"","section5_col5":"","section5_col6":"","section6_col1":"","section6_col2":"","section6_col3":"","section6_col4":"","section6_col5":"","section6_col6":"","select_author":"","test":"","footnotes":""},"categories":[111,1],"tags":[],"class_list":["post-269429","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops","category-blog"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.webscale.com\/wp-json\/wp\/v2\/posts\/269429"}],"collection":[{"href":"https:\/\/www.webscale.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.webscale.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.webscale.com\/wp-json\/wp\/v2\/users\/36"}],"replies":[{"embeddable":true,"href":"https:\/\/www.webscale.com\/wp-json\/wp\/v2\/comments?post=269429"}],"version-history":[{"count":0,"href":"https:\/\/www.webscale.com\/wp-json\/wp\/v2\/posts\/269429\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.webscale.com\/wp-json\/wp\/v2\/media\/269448"}],"wp:attachment":[{"href":"https:\/\/www.webscale.com\/wp-json\/wp\/v2\/media?parent=269429"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.webscale.com\/wp-json\/wp\/v2\/categories?post=269429"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.webscale.com\/wp-json\/wp\/v2\/tags?post=269429"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}