Czy wszyscy zdrowi?
Badanie kondycji aplikacji ASP.NET Core

Mateusz
Turzyński




SRE
Badanie kondycji?
Jak szybko zdiagnozować awarię?
Badanie kondycji pozwala
- Odpowiednio reagować w przypadku awarii
- Budować systemy, które potrafią naprawić się same
Health Checks
na ratunek
https://example.com/healthz
Status: 200 OK
https://example.com/healthz
{
"status":"Unhealthy",
"totalDuration":"00:00:01.1946607",
"entries":{
"sqlserver":{
"data": {
},
"description":"Cannot open server (...) requested by the login.",
"duration":"00:00:01.0638395",
"exception":"Cannot open server (...)",
"status":"Unhealthy",
"tags":[
]
}
}
}Sonda zwracająca szczegółowe dane
Implementacja w ASP.NET Core

Podstawowa konfiguracja
using Microsoft.AspNetCore.Diagnostics.HealthChecks;
var builder = WebApplication.CreateBuilder(args);
// Add services to the container.
builder.Services.AddControllers();
builder.Services.AddHealthChecks();
var app = builder.Build();
app.MapHealthChecks("/healthz");
// Configure the HTTP request pipeline.
app.UseAuthorization();
app.MapControllers();
app.Run();
Dodawanie sond


Dodawanie sond
builder.Services.AddHealthChecks()
.AddSqlServer(builder.Configuration["Data:ConnectionStrings:Sql"])
.AddSqlServer(
connectionString: Configuration["Data:ConnectionStrings:Sql"],
healthQuery: "SELECT 1;",
name: "sql",
tags: new string[] { "db", "sql", "sqlserver" })
.AddRedis(builder.Configuration["Data:ConnectionStrings:Redis"])
.AddAzureKeyVault()
.AddElasticsearch()
.AddMongoDb();
Dostępne sondy



Co jeśli to nie wystarczy?
public class NotFridayHealthcheck : IHealthCheck
{
public Task<HealthCheckResult> CheckHealthAsync(
HealthCheckContext context, CancellationToken cancellationToken = default)
{
if (DateTime.Today.DayOfWeek == DayOfWeek.Friday)
{
return Task.FromResult(HealthCheckResult.Unhealthy("It's friday!"));
}
if (DateTime.Today.DayOfWeek == DayOfWeek.Monday)
{
return Task.FromResult(HealthCheckResult.Degraded("Could be better..."));
}
return Task.FromResult(HealthCheckResult.Healthy("All goood"));
}
}Własna sonda
builder.Services.AddHealthChecks()
.AddCheck<NotFridayHealthcheck>("not-friday")
.AddCheck("lambda-check", () => DateTime.Today.DayOfWeek == DayOfWeek.Friday
? HealthCheckResult.Unhealthy("It's friday!")
: HealthCheckResult.Healthy("All good"));Rejestracja własnej sondy
public static class NotFridayHealthCheckBuilderExtensions
{
private const string DefaultName = "not-friday";
public static IHealthChecksBuilder AddNotFridayHealthCheck(
this IHealthChecksBuilder healthChecksBuilder,
string? name = null,
HealthStatus? failureStatus = null,
IEnumerable<string>? tags = default)
{
return healthChecksBuilder.Add(
new HealthCheckRegistration(
name ?? DefaultName,
_ => new NotFridayHealthcheck(),
failureStatus,
tags));
}
}
builder.Services.AddHealthChecks()
.AddNotFridayHealthCheck();Rejestracja własnej sondy - extension method
Model PULL
vs
Model PUSH
Model PULL
Model PUSH
Własny publisher
public class SampleHealthCheckPublisher : IHealthCheckPublisher
{
public Task PublishAsync(HealthReport report, CancellationToken cancellationToken)
{
if (report.Status == HealthStatus.Healthy)
{
// ...
}
else
{
// ...
}
return Task.CompletedTask;
}
}
builder.Services.Configure<HealthCheckPublisherOptions>(options =>
{
options.Delay = TimeSpan.FromSeconds(2);
options.Period = TimeSpan.FromSeconds(60);
options.Timeout = TimeSpan.FromSeconds(60)
options.Predicate = healthCheck => healthCheck.Tags.Contains("sample");
});
builder.Services.AddSingleton<IHealthCheckPublisher, SampleHealthCheckPublisher>();Gotowe integracje
services
.AddHealthChecks()
.AddApplicationInsightsPublisher()
.AddDatadogPublisher()
.AddPrometheusGatewayPublisher();
Co z tym wszystkim zrobić?

Źródło: https://status.uptimerobot.com/



Źródło: https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks
builder.Services.AddHealthChecksUI(opt =>
{
opt.AddHealthCheckEndpoint("api", "/healthz");
})
.AddInMemoryStorage();
app.MapHealthChecks("/healthz", new HealthCheckOptions
{
Predicate = _ => true,
ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
});
app.MapHealthChecksUI();Konfiguracja HealthChecksUI
HealthChecks a orkiestracja usług

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build-env
WORKDIR /app
COPY *.csproj ./
RUN dotnet restore
COPY . ./
RUN dotnet publish -c Release -o out
FROM mcr.microsoft.com/dotnet/aspnet:6.0
WORKDIR /app
COPY --from=build-env /app/out .
RUN apt-get update && apt-get install -y curl
HEALTHCHECK CMD curl --fail http://localhost/healthz || exit 1
ENTRYPOINT ["dotnet", "HealthCheckSample.Web.dll"]


Kontener startuje
Kontener nie działa
Kontener działa prawidłowo
Jak uleczyć wadliwy kontener?

Docker Autoheal
docker run -d \
--name autoheal \
--restart=always \
-e AUTOHEAL_CONTAINER_LABEL=all \
-v /var/run/docker.sock:/var/run/docker.sock \
willfarrell/autohealIntegracja z Kubernetesem

Kubernetes Probes
- Liveness probe
- Readiness probe
- Startup probe
Kubernetes Probes - konfiguracja
apiVersion: apps/v1
kind: Deployment
metadata:
name: sample-healthcheck-app
spec:
template:
metadata:
labels:
app: sample-healthcheck-app
spec:
containers:
- name: sample-healthcheck-app
image: my-repo/sample-healthcheck-app:1.0
livenessProbe:
httpGet:
path: /healthz
port: 80
initialDelaySeconds: 0
periodSeconds: 10
timeoutSeconds: 1
failureThreshold: 3Różne endpointy na różne sondy
endpoints.MapHealthChecks("/healthz/startup");
endpoints.MapHealthChecks("/healthz",
new HealthCheckOptions
{
Predicate = x => x.Tags.Contains("redis")
});
endpoints.MapHealthChecks("/readyz",
new HealthCheckOptions
{
Predicate = _ => x.Tags.Contains("sql")
}); Azure App Service

Azure Traffic Manager

Zabawa bez końca: integracja z Polly

Retry policy
var retry = Policy
.Handle<SomeExceptionType>()
.WaitAndRetry(new[]
{
TimeSpan.FromSeconds(1),
TimeSpan.FromSeconds(2),
TimeSpan.FromSeconds(3)
});
retry.Execute(() > {} );Circuit Breaker
Circuit Breaker
// WeatherForecastController.cs
public static CircuitBreakerPolicy CircuitBreaker = Policy
.Handle<Exception>()
.CircuitBreaker(3, TimeSpan.FromSeconds(30));
// Get() method
CircuitBreaker.Execute(() =>
{
GetWeather();
});
// Program.cs
builder.Services.AddHealthChecks()
.AddCheck("circuit-breaker",
() => CircuitBreaker.CircuitState == Polly.CircuitBreaker.CircuitState.Closed
? HealthCheckResult.Healthy()
: HealthCheckResult.Unhealthy())Do zapamiętania:
- Healthchecks wspierają monitoring systemu
- Pozwalają odpowiednio reagować
- Mogą wspomóc samonaprawę systemu
- Wspomagają load balancing
- W ASP.NET Core mamy gotową implementację

justjoin.it/sniadanie-z-programowaniem
mateusz.turzynski@aspiresys.com


Czy wszyscy
By Mateusz Turzyński
Czy wszyscy
- 469