{"id":91237,"date":"2020-06-10T11:00:40","date_gmt":"2020-06-10T18:00:40","guid":{"rendered":""},"modified":"2025-12-21T18:44:25","modified_gmt":"2025-12-22T02:44:25","slug":"misconfigured-kubeflow-workloads-are-a-security-risk","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/","title":{"rendered":"Misconfigured Kubeflow workloads are a security risk"},"content":{"rendered":"\n

Azure Security Center (ASC) monitors and defends thousands of Kubernetes clusters running on top of AKS. Azure Security Center regularly searches for and research for new attack vectors against Kubernetes workloads. We recently published a blog post about a large scale campaign<\/a> against Kubernetes clusters that abused exposed Kubernetes dashboards for deploying cryptocurrency miners.<\/p>\n\n\n\n

In this blog, we\u2019ll reveal a new campaign that was observed recently by ASC that targets Kubeflow, a machine learning toolkit for Kubernetes. We observed that this attack effected on tens of Kubernetes clusters.<\/p>\n\n\n\n

Kubeflow is an open-source project, started as a project for running TensorFlow jobs on Kubernetes. Kubeflow has grown and become a popular framework for running machine learning tasks in Kubernetes. Nodes that are used for ML tasks are often relatively powerful, and in some cases include GPUs. This fact makes Kubernetes clusters that are used for ML tasks a perfect target for crypto mining campaigns, which was the aim of this attack.<\/p>\n\n\n\n

During April, we observed deployment of a suspect image from a public repository on many different clusters. The image is ddsfdfsaadfs\/dfsdf:99<\/strong>. By inspecting the image\u2019s layers, we can see that this image runs an XMRIG miner:<\/p>\n\n\n\n

\"We<\/a><\/figure>\n\n\n\n

This repository contains several more images, which differ in the mining configuration. We saw some deployments of those images too.<\/p>\n\n\n\n

Looking at the various clusters that the above image ran on showed that most of them run Kubeflow. This fact implies that the access vector in this attacker is the machine-learning framework.<\/p>\n\n\n\n

The question is how can Kubeflow be used as an access vector for such an attack?<\/p>\n\n\n\n

Kubeflow framework consists of many different services. Some of those services include: frameworks for training models, Katib and Jupyter notebook server, and more.<\/p>\n\n\n\n

Kubeflow is a containerized service: the various tasks run as containers in the cluster. Therefore, if attackers somehow get access to Kubeflow, they have multiple ways to run their malicious image in the cluster.<\/p>\n\n\n\n

The framework is divided into different namespaces, which are a collection of Kubeflow services. Those namespaces are translated into Kubernetes namespaces in which the resources are deployed.<\/p>\n\n\n\n

In first access to Kubeflow, the user is prompted to create a namespace:<\/p>\n\n\n\n

\"In<\/a><\/figure>\n\n\n\n

In the picture above, we created a new namespace with the default name anonymous<\/strong>. This namespace is broadly seen in the attack and was one of the indicators to the access vector in this campaign.<\/p>\n\n\n\n

Kubeflow creates multiple CRDs in the cluster which expose some functionality over the API server:<\/p>\n\n\n\n

\"Kubeflow<\/a><\/figure>\n\n\n\n

In addition, Kubeflow exposes its UI functionality via a dashboard that is deployed in the cluster:<\/p>\n\n\n\n

\"Kubeflow<\/a><\/figure>\n\n\n\n

The dashboard is exposed by Istio ingress gateway, which is by default accessible only internally. Therefore, users should use port-forward to access the dashboard (which tunnels the traffic via the Kubernetes API server).<\/p>\n\n\n\n

In some cases, users modify the setting of the Istio Service to Load-Balancer which exposes the Service (istio-ingressgateway<\/strong> in the namespace istio-system<\/strong>) to the Internet. We believe that some users chose to do it for convenience: without this action, accessing to the dashboard requires tunneling through the Kubernetes API server and isn\u2019t direct. By exposing the Service to the Internet, users can access to the dashboard directly. However, this operation enables insecure access to the Kubeflow dashboard, which allows anyone to perform operations in Kubeflow, including deploying new containers in the cluster.<\/p>\n\n\n\n

If attackers have access to the dashboard, they have multiple methods to deploy a backdoor container in the cluster. We will demonstrate two options:<\/p>\n\n\n\n

    \n
  1. Kubeflow enables users to create a Jupyter notebook server. Kubeflow allows users to choose the image for the notebook server, including an option to specify a custom image:<\/li>\n<\/ol>\n\n\n\n
    \"Image<\/a><\/figure>\n\n\n\n

    This image doesn\u2019t necessarily have to be a legitimate notebook image, thus attackers can run their own image using this feature.<\/p>\n\n\n\n

      \n
    1. Another method that attackers can use is to deploy a malicious container from a real Jupyter notebook: attackers can use a new or existing notebook for running their Python code. The code runs from the notebook server, which is a container by itself with a mounted service account<\/strong>. This service account (by default configuration) has permissions to deploy containers in its namespace. Therefore, attackers can use it to deploy their backdoor container in the cluster. Here\u2019s an example of deploying a container from the notebook using its service account:<\/li>\n<\/ol>\n\n\n\n
      \"Here\u2019s<\/a><\/figure>\n\n\n\n

      The Kubernetes threat matrix<\/a> that we recently published contains techniques that can be used by attackers to attack the Kubernetes cluster. A representation of this campaign in the matrix would look like:<\/p>\n\n\n\n

      \"A<\/a><\/figure>\n\n\n\n

      The attacker used an exposed dashboard (Kubeflow dashboard in this case) for gaining initial access<\/strong> to the cluster. The execution<\/strong> and persistence<\/strong> in the cluster were performed by a container that was deployed in the cluster. The attacker managed to move laterally<\/strong> and deploy the container using the mounted service account. Finally, the attacker impacted<\/strong> the cluster by running a cryptocurrency miner.<\/p>\n\n\n\n

      How to check if your cluster is impacted?<\/h2>\n\n\n\n
        \n
      1. Verify that the malicious container is not deployed in the cluster. The following command can help you to check it:<\/li>\n<\/ol>\n\n\n\n

        kubectl get pods –all-namespaces -o jsonpath=”{.items[*].spec.containers[*].image}”  | grep -i ddsfdfsaadfs<\/em> <\/u><\/strong><\/p>\n\n\n\n

          \n
        1. In case Kubeflow is deployed in the cluster, make sure that its dashboard isn\u2019t exposed to the internet: check the type of the Istio ingress service by the following command and make sure that it is not a load balancer with a public IP:<\/li>\n<\/ol>\n\n\n\n

          kubectl get service istio-ingressgateway -n istio-system<\/em><\/p>\n\n\n\n

          Conclusion<\/h2>\n\n\n\n

          Azure Security Center has detected multiple campaigns against Kubernetes clusters in the past that have a similar access vector: an exposed service to the internet. However, this is the first time that we have identified an attack that targets Kubeflow environments specifically.<\/p>\n\n\n\n

          When deploying a service like Kubeflow within a cluster it is crucial to be aware of security aspects such as:<\/p>\n\n\n\n

            \n
          1. Authentication and access control to the application.<\/li>\n\n\n\n
          2. Monitor the public-facing endpoints of the cluster. Make sure that sensitive interfaces are not exposed to the internet in an unsecure method. You can restrict public load balancers in the cluster by using Azure Policy, which now has integration with Gatekeeper.<\/li>\n\n\n\n
          3. Regularly monitor the runtime environment. This includes monitoring the running containers, their images, and the processes that they run.<\/li>\n\n\n\n
          4. Allow deployments of only trusted images and scan your images for vulnerabilities. The allowed images in the cluster can be restricted by using Azure Policy.<\/li>\n<\/ol>\n\n\n\n

            To learn more about AKS Support in Azure Security Center, please see this documentation<\/a>.<\/p>\n\n\n\n

            Start a trial of Azure Security Center Standard<\/a> to get advanced threat protection capabilities.<\/p>\n","protected":false},"excerpt":{"rendered":"

            Azure Security Center monitors and defends thousands of Kubernetes clusters running on top of Azure Kubernetes Service. In this blog, we\u2019ll reveal a new campaign that was observed recently by ASC that targets Kubeflow, a machine learning toolkit for Kubernetes.<\/p>\n","protected":false},"author":96,"featured_media":91247,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ms_queue_id":[],"ep_exclude_from_search":false,"_classifai_error":"","_classifai_text_to_speech_error":"","_alt_title":"","ms-ems-related-posts":[],"footnotes":""},"post_tag":[3742,3822],"threat-intelligence":[],"content-type":[3662],"job-role":[],"product":[],"topic":[3664,3681],"coauthors":[2290],"class_list":["post-91237","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","tag-azure","tag-microsoft-security-insights","content-type-news","topic-ai-and-machine-learning","topic-risk-management","review-flag-1694638265-576","review-flag-machi-1694638272-641","review-flag-ml-1694638272-297","review-flag-new-1694638263-340"],"yoast_head":"\nMisconfigured Kubeflow workloads are a security risk | Microsoft Security Blog<\/title>\n<meta name=\"description\" content=\"Azure Security Center monitors and defends thousands of Kubernetes clusters running on top of Azure Kubernetes Service. In this blog, we\u2019ll reveal a new campaign that was observed recently by ASC that targets Kubeflow, a machine learning toolkit for Kubernetes.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Misconfigured Kubeflow workloads are a security risk | Microsoft Security Blog\" \/>\n<meta property=\"og:description\" content=\"Azure Security Center monitors and defends thousands of Kubernetes clusters running on top of Azure Kubernetes Service. In this blog, we\u2019ll reveal a new campaign that was observed recently by ASC that targets Kubeflow, a machine learning toolkit for Kubernetes.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/\" \/>\n<meta property=\"og:site_name\" content=\"Microsoft Security Blog\" \/>\n<meta property=\"article:published_time\" content=\"2020-06-10T18:00:40+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-22T02:44:25+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2020\/06\/Misconfigured-Kubeflow.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"440\" \/>\n\t<meta property=\"og:image:height\" content=\"268\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Yossi Weizman\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2020\/06\/Misconfigured-Kubeflow.jpg\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Yossi Weizman\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/\"},\"author\":[{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/author\/yossi-weizman\/\",\"@type\":\"Person\",\"@name\":\"Yossi Weizman\"}],\"headline\":\"Misconfigured Kubeflow workloads are a security risk\",\"datePublished\":\"2020-06-10T18:00:40+00:00\",\"dateModified\":\"2025-12-22T02:44:25+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/\"},\"wordCount\":1098,\"publisher\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2020\/06\/Misconfigured-Kubeflow.jpg\",\"keywords\":[\"Azure\",\"Microsoft Security Insights\"],\"articleSection\":[\"AI and machine learning\",\"Azure Security\",\"Microsoft Defender for Cloud\",\"Threat protection\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/\",\"name\":\"Misconfigured Kubeflow workloads are a security risk | Microsoft Security Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2020\/06\/Misconfigured-Kubeflow.jpg\",\"datePublished\":\"2020-06-10T18:00:40+00:00\",\"dateModified\":\"2025-12-22T02:44:25+00:00\",\"description\":\"Azure Security Center monitors and defends thousands of Kubernetes clusters running on top of Azure Kubernetes Service. In this blog, we\u2019ll reveal a new campaign that was observed recently by ASC that targets Kubeflow, a machine learning toolkit for Kubernetes.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#primaryimage\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2020\/06\/Misconfigured-Kubeflow.jpg\",\"contentUrl\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2020\/06\/Misconfigured-Kubeflow.jpg\",\"width\":440,\"height\":268,\"caption\":\"Tech worker in front a PC wearing headphones.\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Misconfigured Kubeflow workloads are a security risk\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#website\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/\",\"name\":\"Microsoft Security Blog\",\"description\":\"Expert coverage of cybersecurity topics\",\"publisher\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#organization\",\"name\":\"Microsoft Security Blog\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2018\/08\/cropped-cropped-microsoft_logo_element.png\",\"contentUrl\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2018\/08\/cropped-cropped-microsoft_logo_element.png\",\"width\":512,\"height\":512,\"caption\":\"Microsoft Security Blog\"},\"image\":{\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#\/schema\/person\/112e0d6f7a2c99efae36a4a1b2403c2d\",\"name\":\"Teri Seals-Dormer\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/c94dbb0d416ec556cf564fa18bc9ac227ba2ce7b653774bdcc4c631f793fd674?s=96&d=microsoft&r=g03d4afda56ae931a7066d88415e8df14\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/c94dbb0d416ec556cf564fa18bc9ac227ba2ce7b653774bdcc4c631f793fd674?s=96&d=microsoft&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/c94dbb0d416ec556cf564fa18bc9ac227ba2ce7b653774bdcc4c631f793fd674?s=96&d=microsoft&r=g\",\"caption\":\"Teri Seals-Dormer\"},\"url\":\"https:\/\/www.microsoft.com\/en-us\/security\/blog\/author\/v-teseal\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Misconfigured Kubeflow workloads are a security risk | Microsoft Security Blog","description":"Azure Security Center monitors and defends thousands of Kubernetes clusters running on top of Azure Kubernetes Service. In this blog, we\u2019ll reveal a new campaign that was observed recently by ASC that targets Kubeflow, a machine learning toolkit for Kubernetes.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/","og_locale":"en_US","og_type":"article","og_title":"Misconfigured Kubeflow workloads are a security risk | Microsoft Security Blog","og_description":"Azure Security Center monitors and defends thousands of Kubernetes clusters running on top of Azure Kubernetes Service. In this blog, we\u2019ll reveal a new campaign that was observed recently by ASC that targets Kubeflow, a machine learning toolkit for Kubernetes.","og_url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/","og_site_name":"Microsoft Security Blog","article_published_time":"2020-06-10T18:00:40+00:00","article_modified_time":"2025-12-22T02:44:25+00:00","og_image":[{"width":440,"height":268,"url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2020\/06\/Misconfigured-Kubeflow.jpg","type":"image\/jpeg"}],"author":"Yossi Weizman","twitter_card":"summary_large_image","twitter_image":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2020\/06\/Misconfigured-Kubeflow.jpg","twitter_misc":{"Written by":"Yossi Weizman","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#article","isPartOf":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/"},"author":[{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/author\/yossi-weizman\/","@type":"Person","@name":"Yossi Weizman"}],"headline":"Misconfigured Kubeflow workloads are a security risk","datePublished":"2020-06-10T18:00:40+00:00","dateModified":"2025-12-22T02:44:25+00:00","mainEntityOfPage":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/"},"wordCount":1098,"publisher":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#organization"},"image":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#primaryimage"},"thumbnailUrl":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2020\/06\/Misconfigured-Kubeflow.jpg","keywords":["Azure","Microsoft Security Insights"],"articleSection":["AI and machine learning","Azure Security","Microsoft Defender for Cloud","Threat protection"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/","url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/","name":"Misconfigured Kubeflow workloads are a security risk | Microsoft Security Blog","isPartOf":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#primaryimage"},"image":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#primaryimage"},"thumbnailUrl":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2020\/06\/Misconfigured-Kubeflow.jpg","datePublished":"2020-06-10T18:00:40+00:00","dateModified":"2025-12-22T02:44:25+00:00","description":"Azure Security Center monitors and defends thousands of Kubernetes clusters running on top of Azure Kubernetes Service. In this blog, we\u2019ll reveal a new campaign that was observed recently by ASC that targets Kubeflow, a machine learning toolkit for Kubernetes.","breadcrumb":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#primaryimage","url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2020\/06\/Misconfigured-Kubeflow.jpg","contentUrl":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2020\/06\/Misconfigured-Kubeflow.jpg","width":440,"height":268,"caption":"Tech worker in front a PC wearing headphones."},{"@type":"BreadcrumbList","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/2020\/06\/10\/misconfigured-kubeflow-workloads-are-a-security-risk\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/"},{"@type":"ListItem","position":2,"name":"Misconfigured Kubeflow workloads are a security risk"}]},{"@type":"WebSite","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#website","url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/","name":"Microsoft Security Blog","description":"Expert coverage of cybersecurity topics","publisher":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#organization","name":"Microsoft Security Blog","url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2018\/08\/cropped-cropped-microsoft_logo_element.png","contentUrl":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-content\/uploads\/2018\/08\/cropped-cropped-microsoft_logo_element.png","width":512,"height":512,"caption":"Microsoft Security Blog"},"image":{"@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/#\/schema\/person\/112e0d6f7a2c99efae36a4a1b2403c2d","name":"Teri Seals-Dormer","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/c94dbb0d416ec556cf564fa18bc9ac227ba2ce7b653774bdcc4c631f793fd674?s=96&d=microsoft&r=g03d4afda56ae931a7066d88415e8df14","url":"https:\/\/secure.gravatar.com\/avatar\/c94dbb0d416ec556cf564fa18bc9ac227ba2ce7b653774bdcc4c631f793fd674?s=96&d=microsoft&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c94dbb0d416ec556cf564fa18bc9ac227ba2ce7b653774bdcc4c631f793fd674?s=96&d=microsoft&r=g","caption":"Teri Seals-Dormer"},"url":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/author\/v-teseal\/"}]}},"bloginabox_animated_featured_image":null,"bloginabox_display_generated_audio":false,"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Microsoft Security Blog","distributor_original_site_url":"https:\/\/www.microsoft.com\/en-us\/security\/blog","push-errors":false,"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/posts\/91237","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/users\/96"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/comments?post=91237"}],"version-history":[{"count":2,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/posts\/91237\/revisions"}],"predecessor-version":[{"id":144670,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/posts\/91237\/revisions\/144670"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/media\/91247"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/media?parent=91237"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/post_tag?post=91237"},{"taxonomy":"threat-intelligence","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/threat-intelligence?post=91237"},{"taxonomy":"content-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/content-type?post=91237"},{"taxonomy":"job-role","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/job-role?post=91237"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/product?post=91237"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/topic?post=91237"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/wp-json\/wp\/v2\/coauthors?post=91237"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}