Aws-sdk-net: Kesalahan jabat tangan SSL intermiten saat memanggil S3

Perilaku Saat Ini

Kami mengalami kesalahan intermiten ("SSL Handshake gagal dengan kesalahan OpenSSL - SSL_ERROR_ZERO_RETURN") saat berinteraksi dengan klien S3 dari ECS di salah satu aplikasi kami. Kesalahan tampak acak dan sesekali, dan mungkin terjadi saat memuat atau mendapatkan objek dari S3.

Ini adalah contoh pengecualian yang kami dapatkan saat mendapatkan objek dari S3, tepat setelah memuatnya:

{
    "message": "The SSL connection could not be established, see inner exception.",
    "data": {},
    "innerException": {
        "ClassName": "System.Security.Authentication.AuthenticationException",
        "Message": "Authentication failed, see inner exception.",
        "Data": null,
        "InnerException": {
            "message": "SSL Handshake failed with OpenSSL error - SSL_ERROR_ZERO_RETURN.",
            "data": {},
            "stackTrace": "   at Interop.OpenSsl.DoSslHandshake(SafeSslHandle context, Byte[] recvBuf, Int32 recvOffset, Int32 recvCount, Byte[]& sendBuf, Int32& sendCount)
            \n   at System.Net.Security.SslStreamPal.HandshakeInternal(SafeFreeCredentials credential, SafeDeleteContext& context, SecurityBuffer inputBuffer, SecurityBuffer outputBuffer, SslAuthenticationOptions sslAuthenticationOptions)",
            "source": "System.Net.Security",
            "hResult": -2146233088
        },
        "HelpURL": null,
        "StackTraceString": "   at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
        \n   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
        \n   at System.Net.Security.SslState.PartialFrameCallback(AsyncProtocolRequest asyncRequest)\n--- End of stack trace from previous location where exception was thrown ---
        \n   at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
        \n   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
        \n--- End of stack trace from previous location where exception was thrown ---
        \n   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)",
        "RemoteStackTraceString": null,
        "RemoteStackIndex": 0,
        "ExceptionMethod": null,
        "HResult": -2146233087,
        "Source": "System.Private.CoreLib",
        "WatsonBuckets": null
    },
    "stackTrace": "   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
    \n   at System.Net.Http.HttpConnectionPool.CreateConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    \n   at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync(ValueTask`1 creationTask)
    \n   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
    \n   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    \n   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
    \n   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
    \n   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)
    \n   at MyNameSpace.AwsS3Context.GetContentVersionId(Guid id) in /filePath/AwsS3Context.cs:line XX
    \n   at MyNameSpace.AwsS3Context.PutEncryptedContentAsync(Guid id, Stream content) in /filePath/AwsS3Context.cs:line XX
    \n   at MyNameSpace.ManagerClass.SaveContentAsync(Guid id, Stream content) in /filePath/ManagerClass.cs:line XX
    \n   at MyNameSpace.ControllerClass.CreateAsync() in /filePath/ControllerClass.cs:line XX
    \n   at Microsoft.AspNetCore.Mvc.Internal.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
    \n   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeActionMethodAsync()
    \n   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeNextActionFilterAsync()
    \n   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Rethrow(ActionExecutedContext context)
    \n   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
    \n   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync()
    \n   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
    \n   at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
    \n   at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
    \n   at MyNameSpace.APIClass.Startup.<Configure>b__8_1(HttpContext context, Func`1 next) in /filePath/Startup.cs:line XX
    \n   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)",
    "source": "System.Net.Http",
    "hResult": -2146233087
}

Klien S3 dikonfigurasi seperti di bawah ini:

// At startup
services.AddAWSService<IAmazonS3>();

// S3 client class
public class AwsS3Context : IAwsS3Context
{
    private readonly IAmazonS3 awsS3Client;

    public AwsS3Context(IAmazonS3 awsS3Client)
    {
        this.awsS3Client = awsS3Client;
    }

    public async Task<string> GetContentVersionId(Guid id)
    {
        var getS3Request = new GetObjectRequest
                  {
            BucketName = "bucket_name",
            Key = id.ToString()
        };

        using (var s3Item = await awsS3Client.GetObjectAsync(getS3Request).ConfigureAwait(false))
        {
            return s3Item.VersionId;
        }
    } 

    public async Task<string> PutEncryptedContentAsync(Guid id, Stream content)
    {
        var config = new TransferUtilityConfig();

        try
        {
            using (var awsTransferUtility = new TransferUtility(awsS3Client, config))
            {
                var request = new TransferUtilityUploadRequest
                {
                    BucketName = "bucket_name",
                    Key = id.ToString(),
                    InputStream = content,
                    ServerSideEncryptionMethod = ServerSideEncryptionMethod.AES256 
                };

                await awsTransferUtility.UploadAsync(request, CancellationToken.None).ConfigureAwait(false);
                return await GetContentVersionId(Guid);
            }
        }
        finally
        {
            content?.Dispose();
        }
    }
}

Perilaku yang Diharapkan

Kesalahan jabat tangan SSL intermiten saat memanggil S3 tidak diharapkan.

Langkah-langkah untuk Mereproduksi (untuk bug)

Kami tidak dapat mereproduksi kesalahan ini secara konsisten. Untuk menguji, kami mencoba mengisolasi aplikasi di lingkungan dev yang menyediakan satu wadah dan mengirimkannya beban reguler selama beberapa hari, menyesuaikan tingkat dan ukuran permintaan. Setelah ini kami mendapatkan pengecualian yang disebutkan beberapa kali tetapi akar penyebab atau kondisi yang menghasilkan kesalahan tidak terbukti.

Konteks

Ini adalah API sederhana yang diimplementasikan dengan ECS yang menerima berbagai jenis file untuk disimpan secara internal di bucket S3, memberikan id sumber daya kepada kliennya.

Lingkungan Anda

• Versi AWSSDK.Core yang digunakan: AWSSDK.Core (3.3.104.34)
• Perakitan layanan dan versi yang digunakan: AWSSDK.S3 (3.3.110.32)
• Platform .NET yang ditargetkan: .Net Core 2.1 (Microsoft.NETCore.App 2.1.0)
• Versi Visual Studio: VS 2019 (16.4.1)
• Sistem Operasi dan versi: Alpine 3.11 (Dari gambar mcr.microsoft.com/dotnet/core/aspnet:2.1-alpine)

.Info Inti .NET

• Versi .NET Core yang digunakan untuk pengembangan: 2.1
• Versi .NET Core diinstal di lingkungan tempat aplikasi berjalan: 2.1