Sort content of file and remove duplicates

I had to sort contents of a list and remove duplicates..

> cat list.txt | sort -u list-sorted-unique.txt

And then I found that the first column still has duplicates

> cut -d ':' -f1 list-sorted-unique.txt | sort -u | wc -l

Reporting with lsblk and specific columns

lsblk -o NAME,FSTYPE,LABEL,MOUNTPOINT,SIZE,MODEL,SERIAL

NAME              FSTYPE          LABEL    MOUNTPOINT   SIZE MODEL                                    SERIAL
sdf                                                   465.8G WDC  WDBNCE5000P                         190476803028
├─sdf9                                                    8M
└─sdf1            zfs_member      zfs                 465.8G
nvme0n1                                               931.5G Samsung SSD 970 EVO 1TB                  S467NX0KB02478V
├─nvme0n1p1       zfs_member                            128G
└─nvme0n1p2       zfs_member                          803.5G
sdo               btrfs                               931.5G ST1000LM048-2E71                         WDEQTNE0
├─sdo1            zfs_member      zfs_sata            931.5G
└─sdo9            btrfs                                   8M
sdd                                                   465.8G WDC  WDBNCE5000P                         190476801481
├─sdd9                                                    8M
└─sdd1            zfs_member      zfs                 465.8G
sdm               btrfs                               931.5G ST1000LM048-2E71                         WDEN067Y
├─sdm1            zfs_member      zfs_sata            931.5G
└─sdm9            btrfs                                   8M
sdb               isw_raid_member                     465.8G CT500MX500SSD1                           1906E1E8FBC3
└─md126                                               465.8G
  ├─md126p2       LVM2_member                         464.8G
  │ ├─centos-swap swap                                 15.7G
  │ ├─centos-home xfs                      /home      399.1G
  │ └─centos-root xfs                      /             50G
  └─md126p1       xfs                      /boot          1G
sdk                                                     1.8T ST2000LX001-1RG1                         WDZASXRK
├─sdk9                                                    8M
└─sdk1            zfs_member      zfs_sata              1.8T
sdi               btrfs                                 1.8T ST2000LM015-2E81                         WDZAAC2H
├─sdi9            btrfs                                   8M
└─sdi1            zfs_member      zfs_sata              1.8T
sdq                                                   465.8G Samsung SSD 850                          S24CNXAGC07791V
├─sdq9                                                    8M
└─sdq1            zfs_member      zfs                 465.8G
sdg                                                   465.8G WDC  WDBNCE5000P                         190476800512
├─sdg9                                                    8M
└─sdg1            zfs_member      zfs                 465.8G
sde                                                   465.8G WDC  WDBNCE5000P                         190476800250
├─sde9                                                    8M
└─sde1            zfs_member      zfs                 465.8G
sdn               btrfs                               931.5G ST1000LM048-2E71                         WDEMXXPP
├─sdn1            zfs_member      zfs_sata            931.5G
└─sdn9            btrfs                                   8M
sdc                                                   465.8G WDC  WDBNCE5000P                         190476802105
├─sdc9                                                    8M
└─sdc1            zfs_member      zfs                 465.8G
sdl                                                     1.8T ST2000LX001-1RG1                         ZDZ4TJK2
├─sdl1            zfs_member      zfs_sata              1.8T
└─sdl9                                                    8M
nvme1n1                                                 477G PCIe SSD                                 19012351200132
├─nvme1n1p2       zfs_member      zfs                   221G
└─nvme1n1p1       swap                     [SWAP]       256G
sda               isw_raid_member                     465.8G CT500MX500SSD1                           1906E1E8FE6C
└─md126                                               465.8G
  ├─md126p2       LVM2_member                         464.8G
  │ ├─centos-swap swap                                 15.7G
  │ ├─centos-home xfs                      /home      399.1G
  │ └─centos-root xfs                      /             50G
  └─md126p1       xfs                      /boot          1G
sdj               btrfs                                 1.8T ST2000LM015-2E81                         WDZ3WZFN
├─sdj9            btrfs                                   8M
└─sdj1            zfs_member      zfs_sata              1.8T
sdr                                                   465.8G CT500MX500SSD4                           1909E1ED9C29
├─sdr9                                                    8M
└─sdr1            zfs_member      zfs                 465.8G
sdh                                                   465.8G WDC  WDBNCE5000P                         190476800028
├─sdh9                                                    8M
└─sdh1            zfs_member      zfs                 465.8G
sdp               btrfs                               931.5G ST1000LM048-2E71                         WDEPK5PK
├─sdp9            btrfs                                   8M
└─sdp1            zfs_member      zfs_sata            931.5G
 

Temporarily remove and add dedicated SWAP to my Centos

Find out what swap is currently configured
> swapon --show

NAME      TYPE       SIZE  USED PRIO
/dev/dm-1 partition 15.7G 66.9M   -2


Turn off the current swap
> swapoff /dev/dm-1


Find partition for my new swap
> ls -lh /dev/disk/by-id/ |grep nvme-PCIe_SSD_19012351200132-part1


Make the swap
> mkswap /dev/nvme1n1p1


Turn on swap
> swapon /dev/nvme1n1p1

Verify
> swapon --show

NAME           TYPE      SIZE USED PRIO
/dev/nvme1n1p1 partition 256G   0B   -2

Disable ext4 journaling and enable trim ext4 and SSD

Confirm that journaling is enabled and trim mount option is missing (default mount options).
> tune2fs -l /dev/nvme0n1

...
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags:         signed_directory_hash
Default mount options:    user_xattr acl
...

> tune2fs -o discard /dev/nvme1n1
> tune2fs -O ^has_journal /dev/nvme1n1

Note that the ^ character is to toggle between enable/disable

> tune2fs -l /dev/nvme0n1

...
Filesystem features:      ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags:         signed_directory_hash
Default mount options:    user_xattr acl discard
...

Wanted to install mainline kernel 5 on my CentOS

> yum -y install https://www.elrepo.org/elrepo-release-7.0-4.el7.elrepo.noarch.rpm

> rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org

> yum --enablerepo=elrepo-kernel install kernel-ml

> yum -y --enablerepo=elrepo-kernel install kernel-ml-{devel,headers,perf}

After a reboot, I uninstalled all other kernel versions e.g.

> yum remove kernel 

(if you wanted to install a kernel with long term support choose *-lt instead of *-ml) 

Interesting M.2 to PCIe NVME adapter cards & Samsung 970 Evo Plus benchmark

I am actually using two of the EZDIY-FAB cards, the form factor really makes the difference to me, but I also tried the Silverstone and Vantec card

• EZDIY-FAB Dual M.2 Adapter, M.2 PCIe NVMe and PCIe AHCI SSD to PCIe 3.0 x4 and M.2 SATA SSD to SATA III Adapter Card ($19 Amazon)
• Silverstone SST-ECM22 Dual M.2 to PCIe x4 NVMe SSD and SATA 6 G Adapter Card with Advanced Cooling ($33 Newegg)
•  Vantec M.2 NVMe/M.2 SATA SSD PCIe x4 Adapter ($19 Microcenter)

Also going to try the Jeyi card, I am interested to see if the claimed "high" quality components but still lowest price has any impact on performance

• JEYI SK7 M.2 NVMe SSD NGFF To PCI-E X4 Adapter M Key B Key Dual Interface Card Support PCI-E3.0 Dual Voltage 12V+3.3V SATA3 ($9 AliExpress)

Using Iozone, I got these MB/S values for writing a 64GB file on a 500GB Samsung 970 Evo Plus, the values are averages of 6 test runs, mounted in the EZDIY-FAB cards, ext4, CentOS 7, Kernel 5.011-1. No ext4 optimizations enabled.

reclen KB	write	rewrite	read	reread	random read	random write
4	942	915	2647	2849	73	843
16	947	918	3019	3029	231	871
128	913	905	3019	3024	838	859

I think these values are not far off from what public sources report for example: https://www.tomshardware.com/reviews/samsung-970-evo-plus-ssd,5608.html


Bunch of comparable cards..

• $17 RIITOP Dual M.2 to PCIe Adapter, M Key M.2 NVME AHCI SSD to PCI-e 3.0 x4 and B / B+M Key
• $35 SK16 PRO PCIE NVMe M-Key + AHCI B-Key + mSATA Adapter x16 PCI-E 3.0 Full Speed M.2 2280 aluminum
   
• $21 VANTEC UGT-M2PC200 M.2 NVMe + M.2 SATA SSD PCIe X4 Adapter
• $25 Rivo Dual M.2 SATA III and M2 to PCIe 3.0 X4 Adapter Card - Add M.2 SSD Devices to PC or Motherboard
• $19 Dual M.2 PCIe Adapter, M2 SSD NVME (m key) or SATA (b key) 22110 2280 2260 2242 2230 to PCI-e 3.0 x4 Host Controller Expansion Card
• $18 Dual M.2 PCIe Adapter M2 SSD NVME m key or SATA b key 22110 2280 2260 2242 2230 to PCI-e 3.0 x 4 Host Controller Expansion Card with
•  $17 ALLOYSEED PCI-E 4X to NGFF SSD Adapter Card B+M-key M.2 NGFF sata base ssd +NVME pcie ssd

Bunch of specialty cards:

• $299 SuperMicro AOC-SLG3-2E4 NVme PCIe card
• $199 https://www.amazon.com/Supermicro-AOC-SHG3-4M2P-Add-on-Card/dp/B07FYSZR6L
• $89 https://www.delock.de/produkte/G_62704/merkmale.html?setLanguage=en
• ???$ http://amfeltec.com/products/pci-express-gen-3-carrier-board-for-m-2-ssd
• $399 HighPoint SSD7101A-1 4x dedicated 32Gbps M.2 Ports to PCIe 3.0 x16 RAID
  Controller
• $149 https://store.squirrelsresearch.com/sqrl/acorn-nest-x4 (not sure if compatible to SSDs)

EZDIY-FAB Dual M.2 Adapter

Looking to test f2fs with my NVME Flash Drive

> yum install epel-release
> yum --enablerepo=epel-testing install f2fs-tools

Install Glances on CentOS

I really like Glances as a resource monitor for my Linux installation. Here is how to do:

> yum install -y epel-release
> yum install -y python-pip python-devel
> pip install --upgrade pip
> pip install glances

So we can run glances as a web service
> pip install bottle

I think I am going to try using Glances with Grafana: https://www.tecmint.com/install-glances-influxdb-grafana-to-monitor-centos-7/

Installing Iozone on CentOS 7

I followed these instructions: http://repoforge.org/use/


> yum install http://repository.it4i.cz/mirrors/repoforge/redhat/el7/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.3-1.el7.rf.x86_64.rpm

> yum install --enablerepo=rpmforge-extras iozone


Install from sources:

cd /opt

wget http://www.iozone.org/src/current/iozone3_490.tar

tar -xvf iozone3_490.tar
cd iozone3_490/src/current
make linux-AMD64

Getting rid of old CentOS kernels

> yum install yum-utils -y
> package-cleanup --oldkernels --count=1 -y

How to turn off SMB on a Windows 10 installation

A little bit ago I had the need to turn off the SMB service on my Windows 10 installation.


1) How to find what's listening: netstat -n -a | findstr "LISTENING" | findstr ":445".
  TCP    0.0.0.0:445            0.0.0.0:0              LISTENING
  TCP           [::]:445                   [::]:0              LISTENING

2) Start a the Windows power shell with administrative privileges ands execute the following.


$netBTParametersPath = "HKLM:\SYSTEM\CurrentControlSet\Services\NetBT\Parameters"
IF(Test-Path -Path $netBTParametersPath) {
    Set-ItemProperty -Path $netBTParametersPath -Name "SMBDeviceEnabled" -Value 0
}
Set-Service lanmanserver -StartupType Disabled
Stop-Service lanmanserver -Force


3) Restart the computer, use the command from 1) to confirm nothing is listening on part 445 anymore.

How to get the USB Settings and change to PTP mode on a Samsung 8

• Open the phone app of the Samsung Galaxy S8
• Now enter the following code via the keyboard: * # 0808 #
• After you have entered the code, the following screen will appear:
• Under "USB Settings" set the marking on "MTP + ADB". Finished! Then reboot your Samsung Galaxy S8

Source: https://www.solvemix.com/index.php/android-smartphones/1851-samsung-galaxy-s8-usb-settings-for-mtp-adb-how-it-works
Android Smartphones
© Solvemix - quick tips, tricks and solutions

Check out what disks are installed

lshw -class disk