Google Git
Sign in
android / platform / external / grpc-grpc-java / 3432395119095e542a646aeafab6513fb7bf4a1a
commit3432395119095e542a646aeafab6513fb7bf4a1a[log] [tgz]
authorCarl Mastrangelo <[email protected]>Wed Jun 12 11:06:12 2019 -0700
committerGitHub <[email protected]>Wed Jun 12 11:06:12 2019 -0700
treefd4923f30b2639dc4961b11fe99a68a37683dcec
parente4b666aaeba2e9f66812d89250173af0802388aa [diff]
alts: handle inline flushes on close in frame handler

gRPC issues flushes after close in the WriteQueue, which can show up as an NPE in the framer.  This was thought to have been handled, by checking to see if there were any pending writes, but if the close() call gets far enough, the writes will be null.    This causes an NPE when the flush comes though.

The issue is difficult to reproduce, and I think my test case emulates the failure.  EmbeddedChannel is different than the normal Channels we use, making the precise ordering tough.  The test case isn't exactly what the production code would do, but it does have the same ordering.

cc @jiangtaoli2016 

Sample Stack trace:

```
Jun 10, 2019 2:09:03 PM io.grpc.ChannelLogger log
FINEST: [OobChannel<10>] Entering SHUTDOWN state
Jun 10, 2019 2:09:03 PM io.grpc.ChannelLogger log
FINEST: [Subchannel-OOB<11>: (fake-authority-that-is-always-the-same)] NettyClientTransport<14>: (/0:0:0:0:0:0:0:1:20008) SHUTDOWN with UNAVAILABLE(OobChannel is shutdown)
Jun 10, 2019 2:09:03 PM io.grpc.netty.NettyClientHandler close
FINE: Network channel being closed by the application.
Jun 10, 2019 2:09:03 PM io.grpc.internal.ClientCallImpl logIfContextNarrowedTimeout
FINE: Call timeout set to '4999299080' ns, due to context deadline. Explicit call timeout was not set.
Jun 10, 2019 2:09:03 PM io.netty.handler.codec.http2.Http2FrameLogger logGoAway
FINE: [id: 0x4bcebba6, L:/0:0:0:0:0:0:0:1:33296 - R:/0:0:0:0:0:0:0:1:20008] OUTBOUND GO_AWAY: lastStreamId=0 errorCode=0 length=0 bytes=
Jun 10, 2019 2:09:03 PM io.grpc.netty.NettyClientHandler onConnectionError
FINE: Caught a connection error
java.lang.NullPointerException
        at io.grpc.alts.internal.TsiFrameHandler.flush(TsiFrameHandler.java:126)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:754)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:746)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:732)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:201)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:754)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:746)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:732)
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:978)
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:253)
        at io.grpc.netty.WriteQueue.flush(WriteQueue.java:124)
        at io.grpc.netty.WriteQueue.access$000(WriteQueue.java:32)
        at io.grpc.netty.WriteQueue$1.run(WriteQueue.java:44)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:405)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

Jun 10, 2019 2:09:03 PM io.netty.channel.AbstractChannelHandlerContext notifyHandlerException
WARNING: An exception was thrown by a user handler while handling an exceptionCaught event
java.lang.NullPointerException
        at io.grpc.alts.internal.TsiFrameHandler.flush(TsiFrameHandler.java:126)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:754)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:746)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:732)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.onError(Http2ConnectionHandler.java:629)
        at io.grpc.netty.AbstractNettyHandler.exceptionCaught(AbstractNettyHandler.java:81)
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:297)
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:276)
        at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:268)
        at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143)
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:297)
        at io.netty.channel.AbstractChannelHandlerContext.notifyHandlerException(AbstractChannelHandlerContext.java:836)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:756)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:746)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:732)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:201)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:754)
        at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:746)
        at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:732)
        at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:978)
        at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:253)
        at io.grpc.netty.WriteQueue.flush(WriteQueue.java:124)
        at io.grpc.netty.WriteQueue.access$000(WriteQueue.java:32)
        at io.grpc.netty.WriteQueue$1.run(WriteQueue.java:44)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:405)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

Jun 10, 2019 2:09:03 PM io.grpc.netty.NettyClientHandler channelInactive
FINE: Network channel is closed
Jun 10, 2019 2:09:03 PM io.grpc.ChannelLogger log
FINEST: [Subchannel-OOB<11>: (fake-authority-that-is-always-the-same)] NettyClientTransport<14>: (/0:0:0:0:0:0:0:1:20008) Terminated
Jun 10, 2019 2:09:03 PM io.grpc.ChannelLogger log
FINEST: [Subchannel-OOB<11>: (fake-authority-that-is-always-the-same)] Terminated
```
2 files changed
tree: fd4923f30b2639dc4961b11fe99a68a37683dcec
  1. .github/
  2. all/
  3. alts/
  4. android/
  5. android-interop-testing/
  6. api/
  7. auth/
  8. benchmarks/
  9. bom/
  10. buildscripts/
  11. compiler/
  12. context/
  13. core/
  14. cronet/
  15. documentation/
  16. examples/
  17. gae-interop-testing/
  18. gradle/
  19. grpclb/
  20. interop-testing/
  21. netty/
  22. okhttp/
  23. protobuf/
  24. protobuf-lite/
  25. services/
  26. stub/
  27. testing/
  28. testing-proto/
  29. xds/
  30. .bazelignore
  31. .gitattributes
  32. .gitignore
  33. .travis.yml
  34. AUTHORS
  35. BUILD.bazel
  36. build.gradle
  37. codecov.yml
  38. COMPILING.md
  39. CONTRIBUTING.md
  40. gradlew
  41. gradlew.bat
  42. java_grpc_library.bzl
  43. LICENSE
  44. NOTICE.txt
  45. README.md
  46. RELEASING.md
  47. repositories.bzl
  48. run-test-client.sh
  49. run-test-server.sh
  50. SECURITY.md
  51. settings.gradle
  52. WORKSPACE
README.md

gRPC-Java - An RPC library and framework

gRPC-Java works with JDK 7. gRPC-Java clients are supported on Android API levels 14 and up (Ice Cream Sandwich and later). Deploying gRPC servers on an Android device is not supported.

TLS usage typically requires using Java 8, or Play Services Dynamic Security Provider on Android. Please see the Security Readme.

Join the chat at https://gitter.im/grpc/grpc Build Status Coverage Status

Getting Started

For a guided tour, take a look at the quick start guide or the more explanatory gRPC basics.

The examples and the Android example are standalone projects that showcase the usage of gRPC.

Download

Download the JARs. Or for Maven with non-Android, add to your pom.xml:

<dependency>
  <groupId>io.grpc</groupId>
  <artifactId>grpc-netty-shaded</artifactId>
  <version>1.21.0</version>
</dependency>
<dependency>
  <groupId>io.grpc</groupId>
  <artifactId>grpc-protobuf</artifactId>
  <version>1.21.0</version>
</dependency>
<dependency>
  <groupId>io.grpc</groupId>
  <artifactId>grpc-stub</artifactId>
  <version>1.21.0</version>
</dependency>

Or for Gradle with non-Android, add to your dependencies:

compile 'io.grpc:grpc-netty-shaded:1.21.0'
compile 'io.grpc:grpc-protobuf:1.21.0'
compile 'io.grpc:grpc-stub:1.21.0'

For Android client, use grpc-okhttp instead of grpc-netty-shaded and grpc-protobuf-lite instead of grpc-protobuf:

compile 'io.grpc:grpc-okhttp:1.21.0'
compile 'io.grpc:grpc-protobuf-lite:1.21.0'
compile 'io.grpc:grpc-stub:1.21.0'

Development snapshots are available in Sonatypes's snapshot repository.

Generated Code

For protobuf-based codegen, you can put your proto files in the src/main/proto and src/test/proto directories along with an appropriate plugin.

For protobuf-based codegen integrated with the Maven build system, you can use protobuf-maven-plugin (Eclipse and NetBeans users should also look at os-maven-plugin's IDE documentation):

<build>
  <extensions>
    <extension>
      <groupId>kr.motd.maven</groupId>
      <artifactId>os-maven-plugin</artifactId>
      <version>1.5.0.Final</version>
    </extension>
  </extensions>
  <plugins>
    <plugin>
      <groupId>org.xolstice.maven.plugins</groupId>
      <artifactId>protobuf-maven-plugin</artifactId>
      <version>0.5.1</version>
      <configuration>
        <protocArtifact>com.google.protobuf:protoc:3.7.1:exe:${os.detected.classifier}</protocArtifact>
        <pluginId>grpc-java</pluginId>
        <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.21.0:exe:${os.detected.classifier}</pluginArtifact>
      </configuration>
      <executions>
        <execution>
          <goals>
            <goal>compile</goal>
            <goal>compile-custom</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

For protobuf-based codegen integrated with the Gradle build system, you can use protobuf-gradle-plugin:

apply plugin: 'com.google.protobuf'

buildscript {
  repositories {
    mavenCentral()
  }
  dependencies {
    classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.8'
  }
}

protobuf {
  protoc {
    artifact = "com.google.protobuf:protoc:3.7.1"
  }
  plugins {
    grpc {
      artifact = 'io.grpc:protoc-gen-grpc-java:1.21.0'
    }
  }
  generateProtoTasks {
    all()*.plugins {
      grpc {}
    }
  }
}

The prebuilt protoc-gen-grpc-java binary uses glibc on Linux. If you are compiling on Alpine Linux, you may want to use the Alpine grpc-java package which uses musl instead.

API Stability

APIs annotated with @Internal are for internal use by the gRPC library and should not be used by gRPC users. APIs annotated with @ExperimentalApi are subject to change in future releases, and library code that other projects may depend on should not use these APIs.

We recommend using the grpc-java-api-checker (an Error Prone plugin) to check for usages of @ExperimentalApi and @Internal in any library code that depends on gRPC. It may also be used to check for @Internal usage or unintended @ExperimentalApi consumption in non-library code.

How to Build

If you are making changes to gRPC-Java, see the compiling instructions.

High-level Components

At a high level there are three distinct layers to the library: Stub, Channel, and Transport.

Stub

The Stub layer is what is exposed to most developers and provides type-safe bindings to whatever datamodel/IDL/interface you are adapting. gRPC comes with a plugin to the protocol-buffers compiler that generates Stub interfaces out of .proto files, but bindings to other datamodel/IDL are easy and encouraged.

Channel

The Channel layer is an abstraction over Transport handling that is suitable for interception/decoration and exposes more behavior to the application than the Stub layer. It is intended to be easy for application frameworks to use this layer to address cross-cutting concerns such as logging, monitoring, auth, etc.

Transport

The Transport layer does the heavy lifting of putting and taking bytes off the wire. The interfaces to it are abstract just enough to allow plugging in of different implementations. Note the transport layer API is considered internal to gRPC and has weaker API guarantees than the core API under package io.grpc.

gRPC comes with three Transport implementations:

  1. The Netty-based transport is the main transport implementation based on Netty. It is for both the client and the server.
  2. The OkHttp-based transport is a lightweight transport based on OkHttp. It is mainly for use on Android and is for client only.
  3. The in-process transport is for when a server is in the same process as the client. It is useful for testing, while also being safe for production use.