問題：pandas讀取文件過大時內存爆炸
解決：分塊讀取數據再拼接
https://blog.csdn.net/weixin_39750084/article/details/81501395
'''
f = open(path)
data = pd.read_csv(path, sep=',',engine = 'python',iterator=True)
loop = True
chunkSize = 1000
chunks = []
index=0
while loop:
try:
print(index)
chunk = data.get_chunk(chunkSize)
chunks.append(chunk)
index+=1
except StopIteration:
loop = False print("Iteration is stopped.")
print('開始合并')
data = pd.concat(chunks, ignore_index= True)
'''
