{"id":3045,"date":"2021-03-05T00:17:19","date_gmt":"2021-03-04T15:17:19","guid":{"rendered":"http:\/\/matoken.org\/blog\/?p=3045"},"modified":"2021-03-05T00:17:19","modified_gmt":"2021-03-04T15:17:19","slug":"it-seems-that-long-mode-has-become-faster-with-zstd-1-4-9-so-i-will-try-a-little","status":"publish","type":"post","link":"https:\/\/matoken.org\/blog\/2021\/03\/05\/it-seems-that-long-mode-has-become-faster-with-zstd-1-4-9-so-i-will-try-a-little\/","title":{"rendered":"ZSTD 1.4.9\u3067Long Mode\u304c\u901f\u304f\u306a\u3063\u305f\u3089\u3057\u3044\u306e\u3067\u5c11\u3057\u8a66\u3059"},"content":{"rendered":"<div class=\"paragraph\">\n<p>ZSTD 1.4.9\u304c\u30ea\u30ea\u30fc\u30b9\u3055\u308c\u3066\u3044\u307e\u3057\u305f\uff0e<\/p>\n<\/div>\n<div class=\"ulist\">\n<ul>\n<li><a href=\"https:\/\/www.phoronix.com\/scan.php?page=news_item&amp;px=Zstd-1.4.9-Released\">Zstd 1.4.9 Released With ~2x Faster Performance For Long Mode &#8211; Phoronix<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/facebook\/zstd\/releases\/tag\/v1.4.9\">Release Zstandard v1.4.9 \u00b7 facebook\/zstd<\/a><\/li>\n<li><a href=\"https:\/\/facebook.github.io\/zstd\/\">Zstandard &#8211; Real-time data compression algorithm<\/a><\/li>\n<\/ul>\n<\/div>\n<div class=\"paragraph\">\n<p>\u4eca\u56de <code>--long<\/code> \u5229\u7528\u6642\u306b\u6700\u59272\u500d\u306e\u9ad8\u901f\u5316\u304c\u58f2\u308a\u306e\u3088\u3046\u3067\u3059\uff0e\u305d\u308c\u306f\u3059\u3054\u3044\u3063\u3066\u3053\u3068\u3067\u5c11\u3057\u8a66\u3057\u3066\u307f\u307e\u3057\u305f\uff0e<\/p>\n<\/div>\n<div class=\"literalblock\">\n<div class=\"content\">\n<pre>&gt;2x Faster Long Distance Mode\nLong Distance Mode (LDM) --long just got a whole lot faster thanks to optimizations by @mpu in #2483! These optimizations preserve the compression ratio but drastically speed up compression. It is especially noticeable in multithreaded mode, because the long distance match finder is not parallelized.<\/pre>\n<\/div>\n<\/div>\n<div class=\"paragraph\">\n<p>\u73fe\u5728\u306ezstd\u306e\u30d0\u30fc\u30b8\u30e7\u30f3\u306f1\u3064\u524d\u306e1.4.8\u3067\u3057\u305f\uff0e<\/p>\n<\/div>\n<div class=\"listingblock shell\">\n<div class=\"content\">\n<pre class=\"highlight\"><code>$ zstd -V\n*** zstd command line interface 64-bits v1.4.8, by Yann Collet ***<\/code><\/pre>\n<\/div>\n<\/div>\n<div class=\"paragraph\">\n<p>1.4.9\u3092source\u304b\u3089build\u3057\u307e\u3059\uff0e<\/p>\n<\/div>\n<div class=\"listingblock\">\n<div class=\"content\">\n<pre class=\"highlight\"><code class=\"language-shell\" data-lang=\"shell\">$ sudo apt build-dep zstd <b class=\"conum\">(1)<\/b>\n$ git clone https:\/\/github.com\/facebook\/zstd <b class=\"conum\">(2)<\/b>\n$ cd zstd\n$ git checkout v1.4.9 <b class=\"conum\">(3)<\/b>\n$ make <b class=\"conum\">(4)<\/b>\n$ .\/zstd -V <b class=\"conum\">(5)<\/b>\n*** zstd command line interface 64-bits v1.4.9, by Yann Collet ***<\/code><\/pre>\n<\/div>\n<\/div>\n<div class=\"colist arabic\">\n<ol>\n<li>zstd\u30d3\u30eb\u30c9\u306b\u5fc5\u8981\u306a\u30d1\u30c3\u30b1\u30fc\u30b8\u306e\u5c0e\u5165<\/li>\n<li>source\u306eclone<\/li>\n<li>1.4.9\u306b\u5207\u308a\u66ff\u3048<\/li>\n<li>make<\/li>\n<li>\u30d0\u30fc\u30b8\u30e7\u30f3\u78ba\u8a8d<\/li>\n<\/ol>\n<\/div>\n<div class=\"literalblock\">\n<div class=\"title\">ZSTD(1)<\/div>\n<div class=\"content\">\n<pre>       \u2022   --long[=#]:  enables long distance matching with # windowLog, if not # is not present it defaults to 27. This increases the window\n           size (windowLog) and memory usage for both the compressor and decompressor. This setting is designed to  improve  the  compression\n           ratio for files with long matches at a large distance.\n\n           Note: If windowLog is set to larger than 27, --long=windowLog or --memory=windowSize needs to be passed to the decompressor.<\/pre>\n<\/div>\n<\/div>\n<div class=\"paragraph\">\n<p><code>--long<\/code> \u306f 31\u307e\u3067\u306e\u3088\u3046\u3067\u3059\uff0e32\u3092\u6307\u5b9a\u3059\u308b\u3068\u3053\u3046\u3044\u3046\u30a8\u30e9\u30fc\u306b\u306a\u308a\u307e\u3059\uff0e<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p><code>zstd: error 11 : Parameter is out of bound<\/code><\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>\u3061\u3087\u3046\u3069\u5927\u304d\u3081\u306etar\u304c\u3042\u3063\u305f\u306e\u3067\u3053\u308c\u3067\u8a66\u3057\u307e\u3059\uff0e<\/p>\n<\/div>\n<div class=\"listingblock\">\n<div class=\"content\">\n<pre class=\"highlight\"><code class=\"language-shell\" data-lang=\"shell\">$ ls -l ..\/linux-5.11.tar\n-rw-r--r-- 1 matoken matoken 1064212480 Feb 15 18:18 ..\/linux-5.11.tar<\/code><\/pre>\n<\/div>\n<\/div>\n<div class=\"listingblock\">\n<div class=\"content\">\n<pre class=\"highlight\"><code class=\"language-shell\" data-lang=\"shell\">$ grep -m1 ^\"model name\" \/proc\/cpuinfo <b class=\"conum\">(1)<\/b>\nmodel name      : Intel(R) Core(TM) i5-3320M CPU @ 2.60GHz\n$ nproc <b class=\"conum\">(2)<\/b>\n4\n$ \/bin\/time -f\"\\treal\\t%e\" zstd -q -T`nproc` -1 --long=31 ..\/linux-5.11.tar -o \/dev\/null <b class=\"conum\">(3)<\/b>\n        real    8.10\n$ \/bin\/time -f\"\\treal\\t%e\" .\/zstd -q -T`nproc` -1 --long=31 ..\/linux-5.11.tar -o \/dev\/null <b class=\"conum\">(4)<\/b>\n        real    5.27\n$ \/bin\/time -f\"\\treal\\t%e\" zstd -q -T`nproc` --long=31 ..\/linux-5.11.tar -o \/dev\/null <b class=\"conum\">(5)<\/b>\n        real    8.36\n$ \/bin\/time -f\"\\treal\\t%e\" .\/zstd -q -T`nproc` --long=31 ..\/linux-5.11.tar -o \/dev\/null <b class=\"conum\">(6)<\/b>\n        real    5.76\n$ \/bin\/time -f\"\\treal\\t%e\" zstd -q -T`nproc` -19 --long=31 ..\/linux-5.11.tar -o \/dev\/null <b class=\"conum\">(7)<\/b>\n        real    388.15\n$ \/bin\/time -f\"\\treal\\t%e\" .\/zstd -q -T`nproc` -19 --long=31 ..\/linux-5.11.tar -o \/dev\/null <b class=\"conum\">(8)<\/b>\n        real    318.08\n$ \/bin\/time -f\"\\treal\\t%e\" zstd -q -T1 --long=31 ..\/linux-5.11.tar -o \/dev\/null <b class=\"conum\">(9)<\/b>\n        real    11.81\n$ \/bin\/time -f\"\\treal\\t%e\" .\/zstd -q -T1 --long=31 ..\/linux-5.11.tar -o \/dev\/null <b class=\"conum\">(10)<\/b>\n        real    7.89<\/code><\/pre>\n<\/div>\n<\/div>\n<div class=\"colist arabic\">\n<ol>\n<li>CPU\u306f Intel\u00ae Core&#x2122; i5-3320M CPU @ 2.60GHz<\/li>\n<li>\u30d7\u30ed\u30bb\u30c3\u30b5\u6570<\/li>\n<li>\u5727\u7e2e\u30ec\u30d9\u30eb1(\u6700\u4f4e)\u306e1.4.8\u30678.10s<\/li>\n<li>\u5727\u7e2e\u30ec\u30d9\u30eb1(\u6700\u4f4e)\u306e1.4.9\u30675.27s<\/li>\n<li>\u5727\u7e2e\u30ec\u30d9\u30eb3(\u30c7\u30d5\u30a9\u30eb\u30c8)\u306e1.4.8\u30678.36s<\/li>\n<li>\u5727\u7e2e\u30ec\u30d9\u30eb3(\u30c7\u30d5\u30a9\u30eb\u30c8)\u306e1.4.9\u30675.76s<\/li>\n<li>\u5727\u7e2e\u30ec\u30d9\u30eb19(\u6700\u9ad8)\u306e1.4.8\u3067388.15s<\/li>\n<li>\u5727\u7e2e\u30ec\u30d9\u30eb19(\u6700\u9ad8)\u306e1.4.9\u3067318.08s<\/li>\n<li>\u30b9\u30ec\u30c3\u30c9\u6570\u30921\u306b\u3057\u305f\u3068\u304d\u306e\u5727\u7e2e\u30ec\u30d9\u30eb3(\u30c7\u30d5\u30a9\u30eb\u30c8)\u306e1.4.8\u306711.81s<\/li>\n<li>\u30b9\u30ec\u30c3\u30c9\u6570\u30921\u306b\u3057\u305f\u3068\u304d\u306e\u5727\u7e2e\u30ec\u30d9\u30eb3(\u30c7\u30d5\u30a9\u30eb\u30c8)\u306e1.4.8\u30677.89s<\/li>\n<\/ol>\n<\/div>\n<div class=\"paragraph\">\n<p>2\u500d\u3068\u307e\u3067\u306f\u884c\u304d\u307e\u305b\u3093\u304c\u901f\u304f\u306a\u3063\u3066\u3044\u308b\u3088\u3046\u3067\u3059 :)<br \/>\n\u30de\u30eb\u30c1\u30b9\u30ec\u30c3\u30c9\u3067\u805e\u304d\u305d\u3046\u306a\u3053\u3068\u304c\u66f8\u304b\u308c\u3066\u3044\u307e\u3059\u304c\uff0c\u30b9\u30ec\u30c3\u30c9\u6570\u30921\u306b\u3057\u3066\u3082\u3042\u307e\u308a\u5272\u5408\u5909\u308f\u3089\u306a\u3044?<\/p>\n<\/div>\n<div class=\"paragraph\">\n<p>\u3053\u306e\u74b0\u5883\u306f\u3088\u304f\u30b5\u30fc\u30de\u30eb\u30b9\u30ed\u30c3\u30c8\u30ea\u30f3\u30b0\u3057\u3066\u3044\u308b\u306e\u3067\u53c2\u8003\u7a0b\u5ea6\u306b\u2026\u2026\uff0e<\/p>\n<\/div>\n<div class=\"listingblock\">\n<div class=\"title\">\u74b0\u5883<\/div>\n<div class=\"content\">\n<pre>$ .\/zstd -V\n*** zstd command line interface 64-bits v1.4.9, by Yann Collet ***\n$ zstd -V\n*** zstd command line interface 64-bits v1.4.8, by Yann Collet ***\n$ dpkg-query -W zstd\nzstd    1.4.8+dfsg-2.1\n$ lsb_release -dr\nDescription:    Debian GNU\/Linux bullseye\/sid\nRelease:        unstable\n$ uname -m\nx86_64<\/pre>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>ZSTD 1.4.9\u304c\u30ea\u30ea\u30fc\u30b9\u3055\u308c\u3066\u3044\u307e\u3057\u305f\uff0e Zstd 1.4.9 Released With ~2x Faster Performance For Long Mode &#8211; Phoronix Release [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"webmentions_disabled_pings":false,"webmentions_disabled":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":4,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[7,6,199],"tags":[608,607],"class_list":["post-3045","post","type-post","status-publish","format-standard","hentry","category-debian-linux","category-linux","category-sid","tag-zstandard","tag-zstd"],"_links":{"self":[{"href":"https:\/\/matoken.org\/blog\/wp-json\/wp\/v2\/posts\/3045","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/matoken.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/matoken.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/matoken.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/matoken.org\/blog\/wp-json\/wp\/v2\/comments?post=3045"}],"version-history":[{"count":0,"href":"https:\/\/matoken.org\/blog\/wp-json\/wp\/v2\/posts\/3045\/revisions"}],"wp:attachment":[{"href":"https:\/\/matoken.org\/blog\/wp-json\/wp\/v2\/media?parent=3045"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/matoken.org\/blog\/wp-json\/wp\/v2\/categories?post=3045"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/matoken.org\/blog\/wp-json\/wp\/v2\/tags?post=3045"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}