23 Dec 2021

gRPC client-side load balancing in .NET

This blog post is my first contribution to the 2021 C# Advent Calendar many thanks for both Matthew D. Groves and Calvin A. Allen for this opportunity if you interest to read other members blog post so far please check it out I am sure you will find useful blog posts of this Advent Calendar.

In short what is gRPC ?

gRPC is one of the modern open source framework technologies that helps us build applications more efficiently and with high performance. This technology is commonly used in Microservices environments to communicate between components, which depend on each other or call each other, gRPC use the protocol buffers over HTTP/2 for serializing structured data. Fortunately in the .NET ecosystem, we can use gRPC very easily.

Client-side load balancing

Load balancing (LB) allows us to distribute network traffics across many of backend services (instances) to improve the performance and reliability of our applications and it can be categorized into two types:

Layer 4 Transport layer protocols (TCP,UDP) load balancer.
Layer 7 Application layer protocols (HTTP, TLS, DNS) load balancer.

Each of them is use different algorithms to distribute all requests to backend services (instances) the most common algorithm is widely used in above types (Round Robin, Weighted Round Robin,Least Connection,Least response time … etc) in this article we’re try to configure layer 7 as known as client-side load balancer,So, here is the idea client application has a list of backend services IP address or list of DNS service records which has resolved by the client then select an IP address from the list randomly and passing the HTTP requests to the selected address of backend services (instances). in .NET 5 or later versions we can use gRPC client-side load balancing.

THE PROJECT CASE STUDY: In this article we’re going to create a gRPC service app that handling the feed of the .NET Blog and retrieves latest blog post that contains title,summary publishdate and links the gRPC service will parse the feed data into buffer protocol then received by gRPC Client.

Here’s the gRPC service backend.

public class MainFeedService : FeedService.FeedServiceBase
{
private string feedUrl = "https://devblogs.microsoft.com/dotnet/feed/";
private readonly ILogger<MainFeedService> _logger;

public MainFeedService(ILogger<MainFeedService> logger)
{
   _logger = logger;
}


public override async Task<FeedResponce> GetFeed(Empty request, ServerCallContext context)
{

   var httpContext = context.GetHttpContext();

   await context.WriteResponseHeadersAsync(new Metadata { { "host", $"{httpContext.Request.Scheme}://{httpContext.Request.Host}" } });

   FeedResponce listFeedResponce = new();

   using XmlReader reader = XmlReader.Create(feedUrl, new XmlReaderSettings { Async = true });

   SyndicationFeed feed = SyndicationFeed.Load(reader);

   var query = from feedItem in feed.Items.OrderByDescending(x => x.PublishDate)

               select new FeedModel
               {

                   Title = feedItem.Title.Text,
                   Summary = feedItem.Summary.Text,
                   Link = feedItem.Links[0].Uri.ToString(),
                   PublishDate = feedItem.PublishDate.ToString()


               };

   listFeedResponce.FeedItems.AddRange(query.ToList());

   reader.Close();


   return await Task.FromResult(listFeedResponce);
}



}

In the gRPC service we added (host) to MetaData is represented as key,value pairs it’s like an HTTP header, the host key hold host address value for example host = http://localhost:port .

var httpContext = context.GetHttpContext();

await context.WriteResponseHeadersAsync(new Metadata { { "host", $"{httpContext.Request.Scheme}://{httpContext.Request.Host}" } });

The gRPC service backend and client has the following .proto file


syntax = "proto3";

option csharp_namespace = "GrpcServiceK8s";
import "google/protobuf/empty.proto";
package mainservice;



// The feed service definition.
service FeedService {
    rpc GetFeed (google.protobuf.Empty) returns (FeedResponce);
}

// The response message containing list of feed items.
message FeedResponce {
   repeated FeedModel FeedItems = 1;
}


// The response message containing the feed.
message FeedModel {

   string title = 1;
   string summary = 2;
   string link = 3;
   string publishDate = 4;

}

Configure address resolver

On the gRPC client-side to enable load balancing it needs a way to resolve addresses fortunately in .NET provides many ways to configure the gRPC resolver such as :

StaticResolverFactory
Custom Resolver
DnsResolverFactory

StaticResolverFactory

In the static way the resolver doesn’t call addresses from an external source this approach is suitable if we already know all the services address.

Here’s Program.cs file which StaticResolverFactory has registered in dependency injection (DI).

var addressCollection = new StaticResolverFactory(address => new[]
    {
                new DnsEndPoint("localhost", 5244),
                new DnsEndPoint("localhost", 5135),
                new DnsEndPoint("localhost", 5155)
    });


builder.Services.AddSingleton<ResolverFactory>(addressCollection);

builder.Services.AddSingleton(channelService => {

var methodConfig = new MethodConfig
{
    Names = { MethodName.Default },
    RetryPolicy = new RetryPolicy
    {
        MaxAttempts = 5,
        InitialBackoff = TimeSpan.FromSeconds(1),
        MaxBackoff = TimeSpan.FromSeconds(5),
        BackoffMultiplier = 1.5,
        RetryableStatusCodes = { Grpc.Core.StatusCode.Unavailable }
    }
};

     
return GrpcChannel.ForAddress("static:///feed-host", new GrpcChannelOptions

{
    Credentials = ChannelCredentials.Insecure,
    ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new RoundRobinConfig() }, MethodConfigs = { methodConfig } },
    ServiceProvider = channelService

});


});

In the above code (StaticResolverFactory) returns a collection of addresses.

Also we had configured Retry policies (MethodConfig) to provide more availability for the gRPC Client application this section is optional.

Because we used the StaticResolverFactory the schema for the addresses should be a static (static:///feed-host).

gRPC in .NET provide two types of Load Balancing policies (Pick First) and (Round Robin) for the our project we configured the Round Robin policy (Algorithm).

Configure the Round Robin policy:

ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new RoundRobinConfig() }, MethodConfigs = { methodConfig } }

Configure the Pick First policy:

ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new PickFirstConfig() }, MethodConfigs = { methodConfig } }

This policy takes a list of addresses from the resolver then it tries to associate to the list of addresses until it finds a reachable one.

Custom Resolver

Sometimes we need to create a custom thing for problems, fortunately in gRPC .NET it allows us to create a custom resolver, we can implement (Resolver and ResolverFactory) objects to perform this goal. For example we have a xml file (addresses.xml) on our local machine that contains a list of service address.


public class xmlFileResolver : Resolver
{


    private readonly Uri _uriAddress;

    private Action<ResolverResult>? _listener;


    public xmlFileResolver(Uri uriAddress)
    {
        _uriAddress = uriAddress;
    }



    public override async Task RefreshAsync(CancellationToken cancellationToken)
    {


    await Task.Run(async () =>
    {

        using (var reader = new StreamReader(_uriAddress.LocalPath))
        {
            XmlSerializer serializer = new XmlSerializer(typeof(Root));

            var resultsDeserialize = (Root)serializer.Deserialize(reader);

            IReadOnlyList<DnsEndPoint> listAddresses = resultsDeserialize.Element.Select(r => new DnsEndPoint(r.HostName, r.Port)).ToList();

            _listener(ResolverResult.ForResult(listAddresses, serviceConfig: null));


        }


    });

    }


    public override void Start(Action<ResolverResult> listener)
    {
        _listener = listener;
    }
}

   public class customFileResolver : ResolverFactory
{

    public override string Name => "file";

    public override Resolver Create(ResolverOptions options)
    {
        return new xmlFileResolver(options.Address);
    }
}
}

After custom resolver has been created it needs to be registered in the dependency injection (DI) also it needs a path of the file like the following code.

builder.Services.AddSingleton<ResolverFactory, customFileResolver>();
 
builder.Services.AddSingleton(channelService => {

 return GrpcChannel.ForAddress("file:///c:/urls/addresses.xml", new GrpcChannelOptions

    {
        Credentials = ChannelCredentials.Insecure,
        ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new RoundRobinConfig() } },
        ServiceProvider = channelService

    });

});

DnsResolverFactory

In this approach the DNS Resolver will depend on an external source to get Service record that contains address with their ports of the backend service (instance), to configure load balancing we're using the Headless Service in Kubernetes that provides us all requirements for the gRPC client DnsResolver, for a better understanding of the idea we have the following interactive diagram.

“gRPC client-side load balancing in .NET”

First we need to create gRPC service backend pods in the Kubernetes.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: grpc-service-pod
  labels:
    app: grpc-service-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: grpc-service-app
  template:
    metadata:
      labels:
        app: grpc-service-app
    spec:
      containers:
      - name: grpcservice
        image: rebinoq/grpcservicek8s:v1
        resources:
            requests:
              memory: "64Mi"
              cpu: "125m"
            limits:
              memory: "128Mi" 
              cpu: "250m"
        ports:
        - containerPort: 80

kubectl create -f grpc-service-pod.yaml

To create the Headless service in Kubernetes, we run the following yaml code.

note: in the Kubernetes environment to create a Headless service (svc) the clusterIP key must have ‘none’ value.

apiVersion: v1
kind: Service
metadata:
  name: grpc-headless-svc
spec:
  clusterIP: None 
  selector:
    app: grpc-service-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

kubectl create -f grpc-headless-svc.yaml

It is time to check out the status of the Headless service and DNS Resolver we run the following command.

kubectl apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml

kubectl exec -i -t dnsutils -- nslookup grpc-headless-svc       
Server:         10.96.0.10
Address:        10.96.0.10#53

Name:   grpc-headless-svc.default.svc.cluster.local
Address: 10.1.1.163
Name:   grpc-headless-svc.default.svc.cluster.local
Address: 10.1.1.165
Name:   grpc-headless-svc.default.svc.cluster.local
Address: 10.1.1.167

So far every thing going fine, finally we got the service (grpc-headless-svc.default.svc.cluster.local) included the three pods IP address

Again we create another pod for the gRPC client just run the following yaml code.

note: we pass dns:///grpc-headless-svc in the environment variable to the gRPC Client.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: grpc-client-pod
  labels:
    app: grpc-client-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: grpc-client-app
  template:
    metadata:
      labels:
        app: grpc-client-app
    spec:
      containers:
      - name: grpcclient
        image: rebinoq/grpcclientk8s:v1
        resources:
            requests:
              memory: "64Mi"
              cpu: "125m"
            limits:
              memory: "128Mi" 
              cpu: "250m"
        ports:
        - containerPort: 80
        env:
        - name: k8s-svc-url
          value: dns:///grpc-headless-svc

kubectl create -f grpc-client-pod.yaml

The last thing we need to create in Kubernetes is called Service Node this service that enables us to access the gRPC client pod from outside the Kubernetes.

apiVersion: v1
kind: Service
metadata:
  name: grpc-nodeport-svc
spec:
  type: NodePort
  selector:
    app: grpc-client-app
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 80
      nodePort: 30030

kubectl create -f grpc-nodeport-svc.yaml

Because of this port (30030), we can access to the gRPC client app. For example http://localhost:30030

So far every thing has been done on the Kubernetes, let's see the DnsResolver configure code in the gRPC client.

Here’s the dns resolver code

builder.Services.AddSingleton<ResolverFactory>(new DnsResolverFactory(refreshInterval: TimeSpan.FromSeconds(25)));


builder.Services.AddSingleton(channelService => {


    var methodConfig = new MethodConfig
    {
        Names = { MethodName.Default },
        RetryPolicy = new RetryPolicy
        {
            MaxAttempts = 5,
            InitialBackoff = TimeSpan.FromSeconds(1),
            MaxBackoff = TimeSpan.FromSeconds(5),
            BackoffMultiplier = 1.5,
            RetryableStatusCodes = { Grpc.Core.StatusCode.Unavailable }
        }
    };

    var configuration = builder.Configuration;


return GrpcChannel.ForAddress(configuration.GetValue<string>("k8s-svc-url"), new GrpcChannelOptions

{
    Credentials = ChannelCredentials.Insecure,
    ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new RoundRobinConfig() }, MethodConfigs = { methodConfig } },
    ServiceProvider = channelService

});

      
});

Whenever any connection has disconnected the DNS resolver will refresh and immediately it tries to detect another one pod of the gRPS service (instance).

builder.Services.AddSingleton<ResolverFactory>(new DnsResolverFactory(refreshInterval: TimeSpan.FromSeconds(25)));

Here’s the gRPC client code.

public class HomeController : Controller
{
private readonly ILogger<HomeController> _logger;
private readonly GrpcChannel _grpcChannel;
private List<FeedViewModel>? _feedViewModel;
public HomeController(ILogger<HomeController> logger, GrpcChannel grpcChannel)
{
    _logger = logger;
    _grpcChannel = grpcChannel;
            
}

public async Task<IActionResult> Index()
{
             
    var client = new FeedService.FeedServiceClient(_grpcChannel);

    try
    {
        var replay = client.GetFeedAsync(new Google.Protobuf.WellKnownTypes.Empty());


        var responseHeaders = await replay.ResponseHeadersAsync;

        ViewData["hostAdress"] = responseHeaders.GetValue("host");

        var responseFeed = await replay;

                
        _feedViewModel = new List<FeedViewModel>();

        foreach (var item in responseFeed.FeedItems)
        {
                    

            _feedViewModel.Add(new FeedViewModel
            {

                title = item.Title,
                Summary = item.Summary,
                publishDate = DateTime.Parse(item.PublishDate),
                Link = item.Link

            });
        }
    }

    catch (RpcException rpcex)
    {

        _logger.LogWarning(rpcex.StatusCode.ToString());
        _logger.LogError(rpcex.Message);
    }

    return View(_feedViewModel);
}

       
}

The final result of the gRPC load balancing in .NET

“gRPC client-side load balancing in .NET”

In the end of this blog post, I hope you find it useful.

Note: The code only for demonstration purposes!, maybe it not suitable for production use it should be reviewed.

The source code of the project can be found on this GitHub repository.