Redis刪除大Key

原文鏈接:https://www.dubby.cn/detail.html?id=9112

這里說的大key是指包含很多元素的set,sorted set,list和hash。

刪除操作,我們一般想到有2種,delexpire

DEL

Time complexity: O(N) where N is the number of keys that will be removed. When a key to remove holds a value other than a string, the individual complexity for this key is O(M) where M is the number of elements in the list, set, sorted set or hash. Removing a single key that holds a string value is O(1).

如果要刪除的key是一個集合,包含了很多元素,那么DEL時的耗時和元素個數(shù)成正比,所以如果直接DEL,會很慢。

EXPIRE

Note that calling EXPIRE/PEXPIRE with a non-positive timeout or EXPIREAT/PEXPIREAT with a time in the past will result in the key being deleted rather than expired (accordingly, the emitted key event will be del, not expired).

想著expire會不會可以不是直接刪除,可惜官網(wǎng)的描述讓我心灰意冷,如果expire后指定的timeout不是正數(shù),也就是<=0,那其實就是DEL

一點一點刪

我們知道Redis的工作線程是單線程的,如果一個command堵塞了,那所有請求都會超時,這時候,一些騷操作也許可以幫助你。

其實如果想刪除key,可以分解成2個目的,1:不想讓其他人訪問到這個key,2:釋放空間。

那其實我們可以分解成兩步,先用RENAME把原先的key rename成另一個key,比如:

RENAME userInfo:123 "deleteKey:userInfo:123"

然后可以慢慢去刪"deleteKey:userInfo:123",如果是set,那么可以用SREM慢慢刪,最后再用DEL徹底刪掉。

這里可以搞個task去SCAN deleteKey:*,然后慢慢刪除。

UNLINK

Redis 4.0.0提供了一個更加方便的命令

Available since 4.0.0.

Time complexity: O(1) for each key removed regardless of its size. Then the command does O(N) work in a different thread in order to reclaim memory, where N is the number of allocations the deleted objects where composed of.

UNLINK其實是直接返回,然后在后臺線程慢慢刪除。

如果你的Redis版本>=4.0.0,那么強烈建議使用UNLINK來刪除。

刪除耗時測試結(jié)果

單位:微秒

Set個數(shù) DEL EXPIRE UNLINK
1 90 97 75
10 79 67 100
100 51 49 47
1000 303 296 49
10000 2773 2592 52
100000 31210 33157 51
1000000 549388 501536 62
package main

import (
    "github.com/go-redis/redis"
    "fmt"
    "time"
)

func main() {
    client := redis.NewClient(&redis.Options{
        Addr:         "localhost:6379",
        Password:     "",
        DB:           0,
        ReadTimeout:  1000 * 1000 * 1000 * 60 * 60 * 24,
        WriteTimeout: 1000 * 1000 * 1000 * 60 * 60 * 24,
    })

    maxLength := int64(10000 * 100)

    for n := int64(1); n <= maxLength; n *= 10 {
        fmt.Println("Set個數(shù)", n)
        TestDelBigSet(client, n)
        TestExpireBigSet(client, n)
        TestUnlinkBigSet(client, n)
        fmt.Println()
    }
}

func TestDelBigSet(client *redis.Client, count int64) {
    redisKey := fmt.Sprintf("%s%d", "del:", time.Now().Nanosecond())

    for n := int64(0); n < count; n++ {
        err := client.SAdd(redisKey, fmt.Sprintf("%d", n)).Err()
        if err != nil {
            panic(err)
        }
    }

    startTime := CurrentTimestampInMicroSecond()
    client.Del(redisKey)
    endTime := CurrentTimestampInMicroSecond()

    fmt.Println("Del", endTime-startTime)
}

func TestUnlinkBigSet(client *redis.Client, count int64) {
    redisKey := fmt.Sprintf("%s%d", "unlink:", time.Now().Nanosecond())

    for n := int64(0); n < count; n++ {
        err := client.SAdd(redisKey, fmt.Sprintf("%d", n)).Err()
        if err != nil {
            panic(err)
        }
    }
    startTime := CurrentTimestampInMicroSecond()
    client.Unlink(redisKey)
    endTime := CurrentTimestampInMicroSecond()

    fmt.Println("Unlink", endTime-startTime)
}

func TestExpireBigSet(client *redis.Client, count int64) {
    redisKey := fmt.Sprintf("%s%d", "expire:", time.Now().Nanosecond())

    for n := int64(0); n < count; n++ {
        err := client.SAdd(redisKey, fmt.Sprintf("%d", n)).Err()
        if err != nil {
            panic(err)
        }
    }
    startTime := CurrentTimestampInMicroSecond()
    client.Expire(redisKey, 0)
    endTime := CurrentTimestampInMicroSecond()

    fmt.Println("Expire", endTime-startTime)
}

func CurrentTimestampInMicroSecond() int64 {
    return time.Now().UnixNano() / 1000
}

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
平臺聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點,簡書系信息發(fā)布平臺,僅提供信息存儲服務(wù)。

推薦閱讀更多精彩內(nèi)容