Scribd
Upload
83 views

CISUC - Microservices Observability

This internship involves exploring open source monitoring and observability tools to monitor Apache Kafka in a microservices architecture. The objectives are to define a use case using Kafka, identify important metrics and traces to monitor, automatically collect those using OpenCensus and StatsD, send the data to Prometheus and Stackdriver, and define alerts and insights. Over two semesters, the intern would study relevant technologies, define requirements, deploy components, integrate the system, test and evaluate results, and write reports. A 6-month scholarship is available for a BSc graduate.

Uploaded by

ShitzTV
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
83 views

CISUC - Microservices Observability

This internship involves exploring open source monitoring and observability tools to monitor Apache Kafka in a microservices architecture. The objectives are to define a use case using Kafka, identify important metrics and traces to monitor, automatically collect those using OpenCensus and StatsD, send the data to Prometheus and Stackdriver, and define alerts and insights. Over two semesters, the intern would study relevant technologies, define requirements, deploy components, integrate the system, test and evaluate results, and write reports. A 6-month scholarship is available for a BSc graduate.

Uploaded by

ShitzTV
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

# Microservices Observability

- Internship Title: Microservices Observability

- Supervisors:
Filipe Araujo (CISUC/DEI); Jorge Cardoso (CISUC/ Huawei); Jorge Bernardino
(ISEC/IPC)

## Internship

As cloud and micro-service architectures become more complex, with more moving
parts, and highly distributed, there is a need to build deeper levels of
monitoring. Companies face increasing visibility challenges into the health and
performance of their systems. Developing state-of-the-art monitorable systems
requires to understand how recent technological developments and tools fit
together as end-to-end systems and what are the most important metrics and
components to monitor.

Recently, a new term coined observability was introduced to describe a new


paradigm that encompasses the integration and analysis of service logs,
infrastructure and application metrics, and tracing to build new tools for
troubleshooting and analyze complex micro-service systems. Well-known companies
such as Google, Twitter, and Facebook have built internal tools to address the
real need felt by the early adopters of cloud native applications. For example,
Facebook developed Canopy, an end-to-end performance tracing infrastructure,
which records and processes over 1 billion traces per day for performance
analysis.

## Objectives

This internship aims at exploring the use of emergent monitoring and


introspection open source tools available to correctly pinpoint performance
bottlenecks, identify anomalous behavior, and diagnose the root cause of
incidents of micro-service architectures. The main objectives are:

- Define a Use Case using Apache Kafka, a distributed streaming platform, as a


central player to be used throughout the project and to build a Proof of Concept
(PoC).

- Identify which logging, metrics collection, and request tracing should be


monitored from Kafka and which bring a high value to observability.
- Use Google OpenCensus and Etsy StatsD to automatically collect the traces and
metrics previously identified from Apache Kafka.

- Send the collected traces and metrics to Prometheus and Google Stackdriver.

- Use the data collected from the three sources and derive define actionable
alerts and insightful analysis using Prometheus and Google Stackdriver.

The final system build will represent an example of state-of-the-art monitoring


systems. Thus, a study evaluating the performance of the individual components
and the overall solution will be conducted.

## Work Plan

### 1st Semester

- Study the state of the art, namely existing tools, and related technologies
such as logging, metric collection, and tracing (2 months).

- Define a Use Case and derive requirements for advanced observability (1 month)

- Write intermediate report (1 month).

### 2nd Semester

- Deploy base components (Kafka, StatsD, OpenCensus, Prometheus, etc.) (1


months)

- Integrated components (1 months)

- Test and evaluate the results (2 months)

- Write final report (1 month)

## Conditions
This work should take place in the context of a research project funded by FCT.
A 6-month scholarship of 745 euros per month is foreseen for this work for
holders of a B.Sc. degree.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy