Unable to write logs to Elasticsearch from .Net 8 WebAPI using serilog when security is enabled

Question

I am unable to send logs to elasticsearch 8.12.2 when security is enabled on elasticsearch server. However, when I disable the security I can see the logs on elasticsearch server.

Below are the settings from elasticsearch.yml file.

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

I have tried the following with no luck.

This is my program.cs

using Serilog.Sinks.Elasticsearch;
using Serilog;
using System.Reflection;
using Serilog.Exceptions;
using Serilog.Sinks.RollingFile;
using Serilog.Formatting.Json;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.

builder.Services.AddControllers();
// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

configureLogging();
builder.Host.UseSerilog();

var app = builder.Build();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();

app.UseAuthorization();

app.MapControllers();

app.Run();


void configureLogging()
{
    var environment = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT");
    var configuration = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
        .AddJsonFile(
            $"appsettings.{environment}.json", optional: true
        ).Build();

    Log.Logger = new LoggerConfiguration()
        .Enrich.FromLogContext()
        .Enrich.WithExceptionDetails()
        .WriteTo.Elasticsearch(ConfigureElasticSink(configuration, environment))
        .Enrich.WithProperty("Environment", environment)
        .ReadFrom.Configuration(configuration) 
        .CreateLogger();
}



ElasticsearchSinkOptions ConfigureElasticSink(IConfigurationRoot configuration, string environment)
{
      return new ElasticsearchSinkOptions(new Uri(configuration["ElasticConfiguration:Uri"]))
        {
            ModifyConnectionSettings = x => x.BasicAuthentication("elastic", "password"),
            AutoRegisterTemplate = true,
            IndexFormat = $"{Assembly.GetExecutingAssembly().GetName().Name.ToLower().Replace(".", "-")}-{environment.ToLower()}-{DateTime.UtcNow:yyyy-MM}",
            NumberOfReplicas = 1,
            NumberOfShards = 2
        }; 
}

This is my appsettings.json.

"Serilog": {
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft": "Information",
      "System": "Warning"
    }
  }
},
"ElasticConfiguration": {
  "Uri": "https://elastic:[email protected]:9200"
}

I spent few days trying different option but none of these worked. any help would be greatly appreciated.

Bartosz Nowicki · Accepted Answer · 2024-05-12 15:15:42Z

0

I'm just guessing, that you've set up elasticsearch on you own, without trusted certs, so try to turn off cert validation:

ElasticsearchSinkOptions ConfigureElasticSink(IConfigurationRoot configuration, string environment)
{
    return new ElasticsearchSinkOptions(new Uri(configuration["ElasticConfiguration:Uri"]))
    {
        ModifyConnectionSettings = x => x.BasicAuthentication("elastic", "password")
                                       .ServerCertificateValidationCallback((sender, certificate, chain, sslPolicyErrors) => true),

        AutoRegisterTemplate = true,
        IndexFormat = $"{Assembly.GetExecutingAssembly().GetName().Name.ToLower().Replace(".", "-")}-{environment.ToLower()}-{DateTime.UtcNow:yyyy-MM}",
        NumberOfReplicas = 1,
        NumberOfShards = 2
    };
}

answered May 12, 2024 at 15:15

Bartosz Nowicki

161 bronze badge

Sign up to request clarification or add additional context in comments.

1 Comment

Abu Ahmed Over a year ago

Thank you. It was really the certificate issue.

Collectives™ on Stack Overflow

Unable to write logs to Elasticsearch from .Net 8 WebAPI using serilog when security is enabled

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related