下文中所用到的依賴:implementation 'com.squareup.retrofit2:retrofit:2.1.0',雖然用的retrofit的包,但是只用了里面的okhttp部分,因為是demo項目,原先就已經加了依賴了,不方便換了。
現在我們開始多線程分段下載一個大文件,用的是服務上一個apk文件(23M,假設是大文件)
開工,ps:網絡權限,文件讀寫權限,別忘了
1. 既然是多線程下載,先建立一個線程池
ExecutorService executorService = Executors.newFixedThreadPool(5);
2. 確認下載文件的大小
OkHttpClient client = createOkHttpClient();
final Request request = new Request.Builder()
.url(url)
.head()//這里注意請求方式為head
.build();
try {
Response response = client.newCall(request).execute();
httpResponseCall.onResponseSuccess(response);
} catch (IOException e) {
e.printStackTrace();
}
因為本身是線程調用的,這里網絡請求就用同步了,request請求方式為head,只要獲取文件大小就好了,然后在請求返回的head中獲取文件大小
long length = Long.parseLong(response.header("content-length"));//獲取文件長度
long count = length / M + 1;//下載次數 M = 1024 * 1024 * 5,M為一次下載多少長度,這里為5mb
for (int i = 0; i < count; i++) {
long start, end;//每次一次下載的初始位置和結束位置
if (i == count - 1) {
end = length - 1;//因為從0開始的,所以要-1,比如10個字節,就是0-9
} else {
end = (i + 1) * M - 1;
}
start = i * M;
getFile(start, end);//開啟下載線程
}
3. 文件內容下載
OkHttpClient client = createOkHttpClient();
final Request request = new Request.Builder()
//由于只下載文件部分內容,所以如要添加head 格式為 Range,bytes=0-199
.addHeader("Range", String.format("bytes=%d-%d", start, end)
.url(url)
.build();
try {
Response response = client.newCall(request).execute();
httpResponseCall.onResponseSuccess(response);
} catch (IOException e) {
e.printStackTrace();
}
4. 下載之后處理下載數據
InputStream inputStream = response.body().byteStream();//獲取流
FileUtil.appendFileWithInstram(FILE_NAME, inputStream, start);//寫入文件
response.close();
response.notify();
5. 讀取數據寫入文件
public static void appendFileWithInstram(String fileName, InputStream inputStream, long start) {
File file = new File(fileName);//獲取文件,我在下載之前就刪除了原文件
synchronized (TAG) {//設置線程鎖
try {
while (true) {//通過線程鎖,對線程進行排序
long fileLength = PreferenceUtil.getLong(fileName, 0l);//記錄文件長度
JLog.d("FileUtil", "fileLength = " + fileLength +"\n start = "+ start);
if (fileLength == start) {//當文件長度,與下載文件的初始值一樣,則開始寫入文件,否則等待
break;
}
TAG.wait();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
RandomAccessFile randomAccessFile = null;
try {
if (!file.exists()) {
file.createNewFile();
}
randomAccessFile = new RandomAccessFile(file, "rw");
final long M = 1024;
long total = PreferenceUtil.getLong(fileName, 0l);
for (; ; ) {//循環讀取寫入文件
byte[] bytes = new byte[(int) M];
int readCount = inputStream.read(bytes);
if (readCount == 0){//因為請求是一部分,一部分的,不是一次性的,所以會存在0的情況,注意0!=-1
continue;
}
if (readCount == -1) {//讀取完成,結束循環
JLog.d(TAG, "readCount = " + readCount + "\n start = " + start +"\ntotal = " + total);
PreferenceUtil.setLongValue(fileName, total);//將寫入的總長度記錄下來
inputStream.close();
randomAccessFile.close();
break;
}
randomAccessFile.seek(total);//設置偏移量
total += readCount;
//寫入讀取的總長度,不能只寫randomAccessFile.write(bytes)會導致文件長度大于讀取的長度
randomAccessFile.write(bytes, 0, readCount);
JLog.d(TAG, "length = " + randomAccessFile.length());
JLog.d(TAG, "total = " + total);
}
TAG.notifyAll();//結束循環之后,喚醒其他線程進行寫入
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
6. 過程中遇到的問題
02-22 13:37:43.007 7378-7411/com.nmssdmf.testmodule W/System.err: java.net.ProtocolException: unexpected end of stream
02-22 13:37:43.007 7378-7411/com.nmssdmf.testmodule W/System.err: at okhttp3.internal.http.Http1xStream$FixedLengthSource.read(Http1xStream.java:384)
02-22 13:37:43.007 7378-7411/com.nmssdmf.testmodule W/System.err: at okio.RealBufferedSource$1.read(RealBufferedSource.java:386)
02-22 13:37:43.007 7378-7411/com.nmssdmf.testmodule W/System.err: at java.io.InputStream.read(InputStream.java:101)
感覺這個bug是okHttp的,一開始我用直接用response.body.bytes的方式讀取,沒有遇到這個問題,因為直接獲取流的byte數組,所需要的內存占用比較大,后改成從流中獲取的方式,但是不進行線程阻塞,則會在最后一個線程讀取中出現這個問題,經過各種資料查找,在OkHttpClient.Builder
builder.connectTimeout(60, TimeUnit.SECONDS);
builder.readTimeout(60, TimeUnit.SECONDS);
builder.writeTimeout(60, TimeUnit.SECONDS);
大概是因為響應時間的問題,該問題得到解決,再后來把讀寫文件改成現在一樣線程阻塞的方法,在公司網絡下,依舊出現問題(必現),在4g熱點下,沒有該問題。目前還沒有比較好的解決方法。
7. 讀寫文件可以用下面的方法,更方便
/**
* 文件分批寫入
*
* @param fileName 文件路徑
* @param inputStream 需要寫入的內容
* @param start 寫入的初始位置
*/
public synchronized static void appendFileWithInstram(String fileName, InputStream inputStream, long start) {
File file = new File(fileName);
RandomAccessFile randomAccessFile = null;
try {
if (!file.exists()) {
file.createNewFile();
}
randomAccessFile = new RandomAccessFile(file, "rw");
final long M = 1024;
byte[] bytes = new byte[(int) M];
long seek = start;
long total = 0;
for (; ; ) {
int readCount = inputStream.read(bytes);
if (readCount == 0) {
continue;
}
if (readCount == -1) {
JLog.d(TAG, "readCount = " + readCount + "\n start = " + start + "\ntotal = " + total);
JLog.d(TAG, "fileLength = " + randomAccessFile.length()+ "\n seek = " + seek );
inputStream.close();
break;
}
total += readCount;
randomAccessFile.seek(seek );
randomAccessFile.write(bytes, 0, readCount);
seek += readCount;
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}