golang net/http標(biāo)準(zhǔn)庫的client是可以配置各種代理的，http/https/sock5等，不過fasthttp僅支持配置sock5代理，通過定義fasthttp dialfunc實(shí)現(xiàn)：
c := &fasthttp.Client{
Dial: fasthttpproxy.FasthttpSocksDialer("localhost:9050"),
}
項(xiàng)目中碰到的問題是，ops只提供了用squid搭建的http代理，所以是想重新定義一個(gè)http代理的dialfunc，找了fasthttp github倉庫的issue，作者提供了一個(gè)dialFunc
https://github.com/valyala/fasthttp/issues/363#issuecomment-417868528
func FasthttpHTTPDialer(proxyAddr string) fasthttp.DialFunc {
return func(addr string) (net.Conn, error) {
conn, err := fasthttp.Dial(proxyAddr)
if err != nil {
return nil, err
}
req := "CONNECT " + addr + " HTTP/1.1\r\n"
// req += "Proxy-Authorization: xxx\r\n"
req += "\r\n"
if _, err := conn.Write([]byte(req)); err != nil {
return nil, err
}
res := fasthttp.AcquireResponse()
defer fasthttp.ReleaseResponse(res)
res.SkipBody = true
if err := res.Read(bufio.NewReader(conn)); err != nil {
conn.Close()
return nil, err
}
if res.Header.StatusCode() != 200 {
conn.Close()
return nil, fmt.Errorf("could not connect to proxy")
}
return conn, nil
}
}
c := &fasthttp.Client{
Dial: FasthttpHTTPDialer("localhost:9050"),
}
經(jīng)測試，訪問https的站點(diǎn)是OK的，訪問http的站點(diǎn)不行，代理連接不上。先說一下http/https代理的區(qū)別，再說原因。
HTTPS
xx@DESKTOP-TD3VVD0:~$ curl -x 127.0.0.1:1080 https://www.google.com -I -v
* Rebuilt URL to: https://www.google.com/
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 1080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: curl/7.58.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
HTTP/1.1 200 Connection established
<
> HEAD / HTTP/2
> Host: www.google.com
> User-Agent: curl/7.58.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
HTTP/2 200
< date: Sat, 27 Apr 2019 12:06:57 GMT
date: Sat, 27 Apr 2019 12:06:57 GMT
< expires: -1
expires: -1
< cache-control: private, max-age=0
cache-control: private, max-age=0
< content-type: text/html; charset=ISO-8859-1
使用代理訪問https網(wǎng)站時(shí)，會先發(fā)CONNECT請求，讓代理與目標(biāo)站點(diǎn)建立一個(gè)http tunnel，之后在這個(gè)tunnel基礎(chǔ)上進(jìn)行傳輸，對應(yīng)到上面的dialFunc過程就是：
- 客戶端與代理建立一條tcp連接
- 通過這條連接向代理發(fā)出CONNECT請求，讓代理和目標(biāo)站點(diǎn)google建立一條http tunnel，代理返回 HTTP/1.1 200 Connection established，這個(gè)是和普通的http請求返回是不一樣的，CONNECT方法專屬返回
- 之后就可以通過這條tcp連接進(jìn)行請求了
HTTP
xx@DESKTOP-TD3VVD0:~$ curl -x 127.0.0.1:1080 http://www.baidu.com -v -I
* Rebuilt URL to: http://www.baidu.com/
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 1080 (#0)
> HEAD http://www.baidu.com/ HTTP/1.1
> Host: www.baidu.com
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Server: bfe/1.0.8.18
Server: bfe/1.0.8.18
< Date: Sat, 27 Apr 2019 12:15:06 GMT
Date: Sat, 27 Apr 2019 12:15:06 GMT
< Content-Type: text/html
Content-Type: text/html
< Content-Length: 277
Content-Length: 277
< Last-Modified: Mon, 13 Jun 2016 02:50:23 GMT
Last-Modified: Mon, 13 Jun 2016 02:50:23 GMT
< Connection: Close
Connection: Close
< ETag: "575e1f6f-115"
ETag: "575e1f6f-115"
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Pragma: no-cache
Pragma: no-cache
< Accept-Ranges: bytes
Accept-Ranges: bytes
< Proxy-Connection: keep-alive
Proxy-Connection: keep-alive
<
* Closing connection 0
可以看到http站點(diǎn)是不需要發(fā)CONNECT請求的，而是直接將目標(biāo)站點(diǎn)的url作為path 填寫在http請求頭中。
原因
為何上面的dialfunc訪問http站點(diǎn)不行呢，查了squid代理的文檔，發(fā)現(xiàn)squid默認(rèn)會禁止非https站點(diǎn)通過CONNECT方法建立通道，自己搭了個(gè)squid代理去掉配置項(xiàng)，發(fā)現(xiàn)上面的dialfunc是可以訪問http、https站點(diǎn)的，就是說http,https都先建立通過，再請求。
看了fasthttp的源碼，沒辦法在請求前修改request header中的path為目標(biāo)站點(diǎn)url，所以如果需要通過fasthttp使用http代理，那么可以使用上面的dialfunc，同時(shí)代理需要允許非443端口的站點(diǎn)可以建立通道。如果做不到這一點(diǎn)，那么還是建議使用標(biāo)準(zhǔn)庫net/http的client，會更方便一點(diǎn)。
