Iozone & dd on the Gigabyte AORUS NVMe Gen4 1TB SSD

I am going to run a few tests on, there aren't really big performance differences between using ext4 with or without journal. Also using kernel 4.18.0x vs 5.3.3x doesn't make much of a difference.

This runs on an Asus motherboard: https://www.asus.com/us/Motherboards/TUF-GAMING-X570-PLUS/   with the AMD Ryzen 5 3600: https://www.amd.com/en/products/cpu/amd-ryzen-5-3600  and 64 GB of RAM from G-Skill (F4-3200C16-32GTZN)

The test drive is from Gigabyte and 1TB large: https://www.gigabyte.com/us/Solid-State-Drive/AORUS-NVMe-Gen4-SSD-1TB


Iozone: Performance Test of File I/O
Version $Revision: 3.489 $
Compiled for 64 bit mode.
Build: linux-AMD64

Most importantly, I am re-mounting the drive for each test, that clears the buffer cache. Running watch -n 0.2 free -h I can observe the wanted effect.

The exact command used to run the tests is:
./iozone -e -r 4 -r 8 -r 16 -r 32 -r 64 -r 128 -r 512 -r 1024 -r 2048 -r 4096 -r 8192 -r 16384 -s 4g -i 0 -i 1 -i 2 -i 8 -f /apps/tfile -U /apps


CentOS Linux release 8.1.1911 (Core)
4.18.0-147.5.1.el8_1.x86_64

mkfs.ext4 /dev/nvme0n1

						random	random
testfile MB	reclen kB	write	rewrite	read	reread	read	write
4096	4	1609	1892	2391	2291	98	1672
4096	8	1669	1972	2386	2289	167	1831
4096	16	1700	2049	2398	2299	256	1947
4096	32	1720	2075	2395	2302	314	1102
4096	64	1733	2079	2301	2303	503	2048
4096	128	1723	2061	2303	2298	728	2055
4096	512	1659	1953	2406	2309	1747	1969
4096	1024	1657	1975	3458	3405	2176	1968
4096	2048	1661	1972	2304	2307	2796	1961
4096	4096	1669	1964	3879	3876	3255	1955
4096	8192	1668	1952	4143	3866	3732	1948
4096	16384	1605	1898	4082	4079	3824	1901

mkfs.ext4 /dev/nvme0n1

tune2fs -O ^has_journal /dev/nvme0n1

						random	random
testfile MB	reclen kB	write	rewrite	read	reread	read	write
4096	4	1674	1999	2300	2298	97	1734
4096	8	1746	2086	2302	2302	167	1911
4096	16	1787	2124	2302	2302	254	2018
4096	32	1808	2157	2302	2305	314	2105
4096	64	1812	2182	2304	2298	502	2144
4096	128	1801	2153	2305	2299	808	2122
4096	512	1735	2037	2315	2378	1819	2028
4096	1024	1735	2046	3500	3456	2157	2034
4096	2048	1748	2036	2310	2307	2778	2042
4096	4096	1745	2031	3885	3903	3249	2030
4096	8192	1731	2016	4008	3909	3741	2028
4096	16384	1674	1977	4072	4059	3997	1979

mkfs.ext4 /dev/nvme0n1

tune2fs -O ^has_journal /dev/nvme0n1

tune2fs -o ^discard /dev/nvme0n1

						random	random
testfile MB	reclen kB	write	rewrite	read	reread	read	write
4096	4	1659	1980	2394	2261	96	1735
4096	8	1737	2082	2342	2291	117	1922
4096	16	1724	2133	2398	2393	252	2028
4096	32	1805	2157	2398	2298	310	2100
4096	64	1815	2175	2302	2297	493	2133
4096	128	1807	2087	2388	2295	838	2142
4096	512	1736	2048	2439	2313	1756	2038
4096	1024	1736	2035	3471	3325	2115	2047
4096	2048	1732	2043	2403	2404	2680	2050
4096	4096	1745	2046	3879	3896	3260	2045
4096	8192	1739	2027	3975	3884	3727	2032
4096	16384	1675	1978	4071	4055	3983	1985

CentOS Linux release 8.1.1911 (Core)

5.5.3-1.el8.elrepo.x86_64

mkfs.ext4 /dev/nvme0n1

						random	random
testfile MB	reclen kB	write	rewrite	read	reread	read	write
4096	4	1583	1857	2397	2297	97	1647
4096	8	1639	1957	2399	2258	167	1819
4096	16	1687	1981	2386	2385	248	1920
4096	32	1697	2021	2393	2384	316	2000
4096	64	1722	2048	2395	2397	509	2031
4096	128	1711	2010	2397	2298	841	2016
4096	512	1644	1889	2713	2585	1934	1939
4096	1024	1659	1919	3490	3468	2258	1942
4096	2048	1649	1936	2402	2404	2766	1930
4096	4096	1649	1880	3920	3932	3320	1935
4096	8192	1642	1867	4035	3986	3727	1914
4096	16384	1598	1871	4124	4118	4064	1870

mkfs.ext4 /dev/nvme0n1

tune2fs -O ^has_journal /dev/nvme0n1

						random	random
testfile MB	reclen kB	write	rewrite	read	reread	read	write
4096	4	1660	1938	2395	2392	99	1716
4096	8	1723	2025	2397	2389	166	1887
4096	16	1770	2028	2305	2393	255	1997
4096	32	1781	2067	2302	2390	313	2074
4096	64	1800	2143	2396	2395	518	2122
4096	128	1792	2123	2398	2393	800	2122
4096	512	1724	1974	2423	2399	1821	2009
4096	1024	1725	2019	3483	3465	2210	2012
4096	2048	1727	1964	2401	2404	2818	2006
4096	4096	1729	2023	3909	3931	3307	1997
4096	8192	1718	2005	4038	4040	3743	1971
4096	16384	1660	1912	4136	4111	4071	1938

The exact command used to run the tests is:
./iozone -r 4 -r 8 -r 16 -r 32 -r 64 -r 128 -r 512 -r 1024 -r 2048 -r 4096 -r 8192 -r 16384 -s 4g -i 0 -i 1 -i 2 -i 8 -f /apps/tfile -U /apps

The biggest difference here is, not specifying the -e option which then excludes flush (fsync,fflush) in the timing calculations.

mkfs.ext4 /dev/nvme0n1

tune2fs -O ^has_journal /dev/nvme0n1

						random	random
testfile MB	reclen kB	write	rewrite	read	reread	read	write
4096	4	2782	3831	2397	2389	99	2939
4096	8	2997	4164	2397	2390	168	3507
4096	16	3124	4427	2398	2395	259	3974
4096	32	3211	4523	2400	2393	309	4314
4096	64	3296	4589	2398	2395	507	4467
4096	128	3217	4574	2398	2392	799	4537
4096	512	3037	4079	2487	2402	1808	4032
4096	1024	3022	4093	3499	3464	2292	4088
4096	2048	3025	4063	2406	2404	2857	4068
4096	4096	3023	4065	3917	3934	3282	4067
4096	8192	3003	4032	4029	4024	3708	4061
4096	16384	2836	3804	4120	4110	3885	3817

dd if=/dev/zero of=/apps/file1.txt count=50240 bs=1M conv=fsync
 50240+0 records in
50240+0 records out
52680458240 bytes (53 GB, 49 GiB) copied, 26.8013 s, 2.0 GB/s

dd if=/dev/zero of=/apps/file2.txt count=50240 bs=1M
50240+0 records in
50240+0 records out
52680458240 bytes (53 GB, 49 GiB) copied, 26.3247 s, 2.0 GB/s