Http(s)URLConnection背后的惊人真相

发表于

1.疑问

前些天有朋友问了我一个问题：对于像Http(s)URLConnection这样Android官方提供的用于进行网络连接的类，在进行很好的封装后，使用起来也很方便，但问题来了，使用它们是否足够可靠？

由于发现有一些基于HttpURLConnection的网络请求框架在github上已经收获了很多的star, 可能现在也有不少人在项目中采用了它们，但是在经过深入地查看源码之后发现，这样做是有很大的安全隐患的，所以特意写了本篇文章，希望看到本文的童鞋能够尽快切换到使用最新的okhttp版本。

2.分析

相信很多人都知道，从Android 4.4开始，Http(s)URLConnection的底层实现就使用okhttp了，但是具体是如何实现的呢？
毕竟Http(s)URLConnection对外的接口和okhttp对外的接口不一样，这样的话就必然存在一些类作为中间桥梁，将HttpURLConnection的方法调用转换为对okhttp的调用。

由于HttpsURLConnection和HttpURLConnection的使用大同小异，所以这里就以HttpURLConnection为例进行讲解，另外，由于Android 5.0-Android 7.0中都是基于okhttp 2.x版本的，从而具体的调用差别不大，这里就以Android 5.0的源码为例进行讲解。

2.1 HttpURLConnection的请求过程

HttpURLConnection对外提供的接口确实是很方便使用，一个典型的使用HttpURLConnection进行网络请求的过程如下:

其中蓝色标注的是过程中的重要方法调用，从图中可以看出来，其实无非也就是设置连接参数，写入头部参数，建立连接，获得输出流之后将请求体写入，之后获得服务器的响应流，将响应流再转换为响应实体(比如JSON对象,文件等)。

那么问题来了，在我们调用HttpURLConnection的这些方法时，它是如何将其转换为okhttp的方法调用的？

下面将从okhttp的角度进行详细的拆解。

2.2 结合okhttp进行拆解

在拆解之前，有必要先了解一下源码的结构，需要注意的是HttpURLConnection相关的源码在不同的Android版本上可能会不一样，比如在Android 5.x上是在/libcore/luni/src/main/java/java/net下的,而在Android 7.x上是在/libcore/ojluni/src/main/java/java/net下面。

而okhttp的源码，则一直是在/external/okhttp/下。

架构其实很简单，UML图如下所示:

了解了大致的架构，再去拆解就很简单了。

2.2.1 URL.openConnection()的拆解

首先是URL.openConnection的拆解，该方法如下:

1
2
3
public URLConnection openConnection() throws IOException {
     return streamHandler.openConnection(this);
 }

这个方法非常简单，所以看一下streamHandler对象是什么，看一下setupStreamHandler()方法: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
void setupStreamHandler() {
       // Check for a cached (previously looked up) handler for
       // the requested protocol.
       streamHandler = streamHandlers.get(protocol);
       if (streamHandler != null) {
           return;
       }
       // If there is a stream handler factory, then attempt to
       // use it to create the handler.
       if (streamHandlerFactory != null) {
           streamHandler = streamHandlerFactory.createURLStreamHandler(protocol);
           if (streamHandler != null) {
               streamHandlers.put(protocol, streamHandler);
               return;
           }
       }
       // Check if there is a list of packages which can provide handlers.
       // If so, then walk this list looking for an applicable one.
       String packageList = System.getProperty("java.protocol.handler.pkgs");
       ClassLoader contextClassLoader = Thread.currentThread().getContextClassLoader();
       if (packageList != null && contextClassLoader != null) {
           for (String packageName : packageList.split("\\|")) {
               String className = packageName + "." + protocol + ".Handler";
               try {
                   Class<?> c = contextClassLoader.loadClass(className);
                   streamHandler = (URLStreamHandler) c.newInstance();
                   if (streamHandler != null) {
                       streamHandlers.put(protocol, streamHandler);
                   }
                   return;
               } catch (IllegalAccessException ignored) {
               } catch (InstantiationException ignored) {
               } catch (ClassNotFoundException ignored) {
               }
           }
       }
       // Fall back to a built-in stream handler if the user didn't supply one
       if (protocol.equals("file")) {
           streamHandler = new FileHandler();
       } else if (protocol.equals("ftp")) {
           streamHandler = new FtpHandler();
       } else if (protocol.equals("http")) {
           try {
               String name = "com.android.okhttp.HttpHandler";
               streamHandler = (URLStreamHandler) Class.forName(name).newInstance();
           } catch (Exception e) {
               throw new AssertionError(e);
           }
       } else if (protocol.equals("https")) {
           try {
               String name = "com.android.okhttp.HttpsHandler";
               streamHandler = (URLStreamHandler) Class.forName(name).newInstance();
           } catch (Exception e) {
               throw new AssertionError(e);
           }
       } else if (protocol.equals("jar")) {
           streamHandler = new JarHandler();
       }
       if (streamHandler != null) {
           streamHandlers.put(protocol, streamHandler);
       }
   }

显然，这里是通过反射调用com.android.okhttp.HttpHandler来新建对象，但关键是:整个Android源码中根本找不到com.android.okhttp.HttpHandler这个类!!!
当时这个问题确实困扰了好一会儿，后面才发现其实是因为jarjar-rules的存在，其实路径为/external/okhttp/jarjar-rules.txt,内容如下:

1
2
rule com.squareup.** com.android.@1
rule okio.** com.android.okio.@1

这表示com.squareup开关的包会在编译时打包成com.android开头的包，所以这里的com.android.okhttp.HttpHandler对应的源码其实是com.squareup.okhttp.HttpHandler,查找发现这个类是存在的，而它openConnection()方法如下:

1
2
3
protected URLConnection openConnection(URL url) throws IOException {
        return newOkHttpClient(null /* proxy */).open(url);
    }

到这里就看到OkHttpClient了，而实际上OkHttpClient中本来没有open()这个方法的，显然这是ASOP团队添加的，方法如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
public HttpURLConnection open(URL url) {
  return open(url, proxy);
}
HttpURLConnection open(URL url, Proxy proxy) {
  String protocol = url.getProtocol();
  OkHttpClient copy = copyWithDefaults();
  copy.proxy = proxy;
  if (protocol.equals("http")) return new HttpURLConnectionImpl(url, copy);
  if (protocol.equals("https")) return new HttpsURLConnectionImpl(url, copy);
  throw new IllegalArgumentException("Unexpected protocol: " + protocol);
}

可见，URL.openConnection()方法拆解开来如下:

2.2.2 HttpURLConnection.connect()的拆解

从前面的分析可知，URL.openConnection()最终返回的是HttpURLConnectionImpl对象，所以HttpURLConnection.connect()其实就是HttpURLConnectionImpl.connect(),该方法如下:

1
2
3
4
5
6
7
@Override public final void connect() throws IOException {
   initHttpEngine();
   boolean success;
   do {
     success = execute(false);
   } while (!success);
 }

这里其实就是初始化HttpEngine和调用execute()方法，该方法如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
private boolean execute(boolean readResponse) throws IOException {
   try {
     httpEngine.sendRequest();
     route = httpEngine.getRoute();
     handshake = httpEngine.getConnection() != null
         ? httpEngine.getConnection().getHandshake()
         : null;
     if (readResponse) {
       httpEngine.readResponse();
     }
     return true;
   } catch (IOException e) {
     HttpEngine retryEngine = httpEngine.recover(e);
     if (retryEngine != null) {
       httpEngine = retryEngine;
       return false;
     }
     // Give up; recovery is not possible.
     httpEngineFailure = e;
     throw e;
   }
 }

这里的recover()其实也一样埋下了我在专栏上一篇文章中提到的无限重试的隐患，这个后面会再说。
显然，在这里才真正地发送请求并和服务端建立连接。所以从okhttp的角度来年HttpURLConnection.connect()方法的调用过程如下:

2.2.3 HttpURLConnection.getOutputStream()的拆解

类似地，看HttpURLConnectionImpl的getOutputStream()方法:

1
2
3
4
5
6
7
8
9
10
11
12
@Override public final OutputStream getOutputStream() throws IOException {
   connect();
   BufferedSink sink = httpEngine.getBufferedRequestBody();
   if (sink == null) {
     throw new ProtocolException("method does not support a request body: " + method);
   } else if (httpEngine.hasResponse()) {
     throw new ProtocolException("cannot write request body after response has been read");
   }
   return sink.outputStream();
 }

由于连接复用，所以这里其实并不会真的再次创建连接，所以这里其实关键就是httpEngine.getBufferedRequestBody()了，这个方法如下: 

1
2
3
4
5
6
7
8
public final BufferedSink getBufferedRequestBody() {
  BufferedSink result = bufferedRequestBody;
  if (result != null) return result;
  Sink requestBody = getRequestBody();
  return requestBody != null
      ? (bufferedRequestBody = Okio.buffer(requestBody))
      : null;
}

显然是调用了getRequestBody()，而getRequestBody()方法如下: 

1
2
3
4
5
/** Returns the request body or null if this request doesn't have a body. */
 public final Sink getRequestBody() {
   if (responseSource == null) throw new IllegalStateException();
   return requestBodyOut;
 }

这个方法的注释写得很清楚了，就是返回请求体，如果这个请求根本没有body的话就返回null,而requestBodyOut的赋值其实是发生在HttpEngine.sendRequest()中，如下: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
public final void sendRequest() throws IOException {
    if (responseSource != null) return; // Already sent.
    if (transport != null) throw new IllegalStateException();
    Request request = networkRequest(userRequest);
    OkResponseCache responseCache = client.getOkResponseCache();
    Response cacheCandidate = responseCache != null
        ? responseCache.get(request)
        : null;
    long now = System.currentTimeMillis();
    CacheStrategy cacheStrategy = new CacheStrategy.Factory(now, request, cacheCandidate).get();
    responseSource = cacheStrategy.source;
    networkRequest = cacheStrategy.networkRequest;
    cacheResponse = cacheStrategy.cacheResponse;
    if (responseCache != null) {
      responseCache.trackResponse(responseSource);
    }
    if (cacheCandidate != null
        && (responseSource == ResponseSource.NONE || cacheResponse == null)) {
      closeQuietly(cacheCandidate.body()); // The cache candidate wasn't applicable. Close it.
    }
    if (networkRequest != null) {
      // Open a connection unless we inherited one from a redirect.
      if (connection == null) {
        connect(networkRequest);
      }
      // Blow up if we aren't the current owner of the connection.
      if (connection.getOwner() != this && !connection.isSpdy()) throw new AssertionError();
      transport = (Transport) connection.newTransport(this);
      // Create a request body if we don't have one already. We'll already have
      // one if we're retrying a failed POST.
      if (hasRequestBody() && requestBodyOut == null) {
        requestBodyOut = transport.createRequestBody(request);
      }
    } else {
      // We're using a cached response. Recycle a connection we may have inherited from a redirect.
      if (connection != null) {
        client.getConnectionPool().recycle(connection);
        connection = null;
      }
      // No need for the network! Promote the cached response immediately.
      this.userResponse = cacheResponse.newBuilder()
          .request(userRequest)
          .priorResponse(stripBody(priorResponse))
          .cacheResponse(stripBody(cacheResponse))
          .build();
      if (userResponse.body() != null) {
        initContentStream(userResponse.body().source());
      }
    }
  }

注意其中的requestBodyOut=transport.createRequestBody(request); 而一旦建立了输出流，就可以往里面写入请求体了。所以从okhttp的角度来看，HttpURLConnection.getOutputStream()的调用过程异常简单:

#####2.2.4 HttpURLConnection.getInputStream()的拆解
在往输出流中写入请求体之后，下一步就是获取服务端的响应，即获得输入流。类似地，查看HttpURLConnectionImpl.getInputStream()的实现，如下: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
@Override public final InputStream getInputStream() throws IOException {
    if (!doInput) {
      throw new ProtocolException("This protocol does not support input");
    }
    HttpEngine response = getResponse();
    // if the requested file does not exist, throw an exception formerly the
    // Error page from the server was returned if the requested file was
    // text/html this has changed to return FileNotFoundException for all
    // file types
    if (getResponseCode() >= HTTP_BAD_REQUEST) {
      throw new FileNotFoundException(url.toString());
    }
    InputStream result = response.getResponseBodyBytes();
    if (result == null) {
      throw new ProtocolException("No response body exists; responseCode=" + getResponseCode());
    }
    return result;
  }

如果连接正常的话，此时的HttpEngine中已经有了服务端的响应了，下面看HttpEngine.getResponseBodyBytes()方法:

1
2
3
4
5
6
public final InputStream getResponseBodyBytes() {
   InputStream result = responseBodyBytes;
   return result != null
       ? result
       : (responseBodyBytes = Okio.buffer(getResponseBody()).inputStream());
 }

注意其中的getResponseBody()方法：

1
2
3
4
public final Source getResponseBody() {
    if (userResponse == null) throw new IllegalStateException();
    return responseBody;
  }

而responseBody的赋值在HttpEngine.readResponse()方法中，如下: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
public final void readResponse() throws IOException {
   if (userResponse != null) {
     return; // Already ready.
   }
   if (networkRequest == null && cacheResponse == null) {
     throw new IllegalStateException("call sendRequest() first!");
   }
   if (networkRequest == null) {
     return; // No network response to read.
   }
   // Flush the request body if there's data outstanding.
   if (bufferedRequestBody != null && bufferedRequestBody.buffer().size() > 0) {
     bufferedRequestBody.flush();
   }
   if (sentRequestMillis == -1) {
     if (OkHeaders.contentLength(networkRequest) == -1
         && requestBodyOut instanceof RetryableSink) {
       // We might not learn the Content-Length until the request body has been buffered.
       long contentLength = ((RetryableSink) requestBodyOut).contentLength();
       networkRequest = networkRequest.newBuilder()
           .header("Content-Length", Long.toString(contentLength))
           .build();
     }
     transport.writeRequestHeaders(networkRequest);
   }
   if (requestBodyOut != null) {
     if (bufferedRequestBody != null) {
       // This also closes the wrapped requestBodyOut.
       bufferedRequestBody.close();
     } else {
       requestBodyOut.close();
     }
     if (requestBodyOut instanceof RetryableSink) {
       transport.writeRequestBody((RetryableSink) requestBodyOut);
     }
   }
   transport.flushRequest();
   networkResponse = transport.readResponseHeaders()
       .request(networkRequest)
       .handshake(connection.getHandshake())
       .header(OkHeaders.SENT_MILLIS, Long.toString(sentRequestMillis))
       .header(OkHeaders.RECEIVED_MILLIS, Long.toString(System.currentTimeMillis()))
       .setResponseSource(responseSource)
       .build();
   connection.setHttpMinorVersion(networkResponse.httpMinorVersion());
   receiveHeaders(networkResponse.headers());
   if (responseSource == ResponseSource.CONDITIONAL_CACHE) {
     if (cacheResponse.validate(networkResponse)) {
       userResponse = cacheResponse.newBuilder()
           .request(userRequest)
           .priorResponse(stripBody(priorResponse))
           .headers(combine(cacheResponse.headers(), networkResponse.headers()))
           .cacheResponse(stripBody(cacheResponse))
           .networkResponse(stripBody(networkResponse))
           .build();
       transport.emptyTransferStream();
       releaseConnection();
       // Update the cache after combining headers but before stripping the
       // Content-Encoding header (as performed by initContentStream()).
       OkResponseCache responseCache = client.getOkResponseCache();
       responseCache.trackConditionalCacheHit();
       responseCache.update(cacheResponse, stripBody(userResponse));
       if (cacheResponse.body() != null) {
         initContentStream(cacheResponse.body().source());
       }
       return;
     } else {
       closeQuietly(cacheResponse.body());
     }
   }
   userResponse = networkResponse.newBuilder()
       .request(userRequest)
       .priorResponse(stripBody(priorResponse))
       .cacheResponse(stripBody(cacheResponse))
       .networkResponse(stripBody(networkResponse))
       .build();
   if (!hasResponseBody()) {
     // Don't call initContentStream() when the response doesn't have any content.
     responseTransferSource = transport.getTransferStream(storeRequest);
     responseBody = responseTransferSource;
     return;
   }
   maybeCache();
   initContentStream(transport.getTransferStream(storeRequest));
 }

注意其中的initContentStream(),它就是用于初始化输入流的:

1
2
3
4
5
6
7
8
9
10
11
12
private void initContentStream(Source transferSource) throws IOException {
    responseTransferSource = transferSource;
    if (transparentGzip && "gzip".equalsIgnoreCase(userResponse.header("Content-Encoding"))) {
      userResponse = userResponse.newBuilder()
          .removeHeader("Content-Encoding")
          .removeHeader("Content-Length")
          .build();
      responseBody = new GzipSource(transferSource);
    } else {
      responseBody = transferSource;
    }
  }

而这个输入流来自于Transport.getTransferStream()方法,如果是http协议的话，则实现类为HttpTransport,该方法如下: 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
  @Override public Source getTransferStream(CacheRequest cacheRequest) throws IOException {
    if (!httpEngine.hasResponseBody()) {
      return httpConnection.newFixedLengthSource(cacheRequest, 0);
    }
    if ("chunked".equalsIgnoreCase(httpEngine.getResponse().header("Transfer-Encoding"))) {
      return httpConnection.newChunkedSource(cacheRequest, httpEngine);
    }
    long contentLength = OkHeaders.contentLength(httpEngine.getResponse());
    if (contentLength != -1) {
      return httpConnection.newFixedLengthSource(cacheRequest, contentLength);
    }
    // Wrap the input stream from the connection (rather than just returning
    // "socketIn" directly here), so that we can control its use after the
    // reference escapes.
    return httpConnection.newUnknownLengthSource(cacheRequest);
  }
}

所以，HttpURLConnection.getInputStream()方法拆解开如下:

3.结论

由前一节的分析可知，从Android 4.4开始，HttpURLConnection的实现确实是通过调用okhttp完成的，而具体的方法则是通过HttpHandler这个桥梁，以及在OkHttpClient, HttpEngine中增加相应的方法来实现，当然，其实还涉及一些类的增加或删除，但是由于不是核心类，这里就不再赘述了。

表面来看，似乎这一切都没什么问题，但是，由于其实现是调用okhttp,那么okhttp有的bug它其实一个也不少(比如HttpEngine中的recover()方法其实一样也会有无限重试的隐患），那我们来看看Android各版本引用的okhttp版本是多少呢!

  • Android 4.4.4_r1: 1.1.2
  • Android 5.0.1_r1: 2.0.0
  • Android 6.0.1_r1: 2.4.0
  • Android 7.1.0_r1: 2.6.0

看到Android 7.1.0还在使用okhttp 2.6.0的时候，心中有一万头草泥马呼啸而过。8.0的源码暂时还下载不到，所以暂时还不知道它采用了什么版本，不过几乎可以肯定是没有跟上主流的okhttp版本，所以结论就很明显了:

使用Http(s)URLConnection进行网络请求是极其不可靠的，除非你想把okhttp从1.1.2以来到现在的bug再经历一遍，而且是在不同的手机版本上经历不一样的bug!