wistor@wistor-003:~$ sudo apt-get install ntop [sudo] password for wistor: Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: javascript-common libdbi1 libjs-mochikit librrd4 ntop-data python-mako python-markupsafe ttf-dejavu ttf-dejavu-extra wwwconfig-common Suggested packages: graphviz gsfonts geoip-database-contrib python-beaker python-mako-doc mysql-client postgresql-client apache2 The following NEW packages will be installed: javascript-common libdbi1 libjs-mochikit librrd4 ntop ntop-data python-mako python-markupsafe ttf-dejavu ttf-dejavu-extra wwwconfig-common 0 upgraded, 11 newly installed, 0 to remove and 1 not upgraded. Need to get 5,864 kB of archives. After this operation, 16.8 MB of additional disk space will be used. Do you want to continue [Y/n]? y Get:1 http://tw.archive.ubuntu.com/ubuntu/ oneiric/universe wwwconfig-common all 0.2.2 [18.0 kB] Get:2 http://tw.archive.ubuntu.com/ubuntu/ oneiric/universe javascript-common all 8 [4,208 B] Get:3 http://tw.archive.ubuntu.com/ubuntu/ oneiric/main libdbi1 amd64 0.8.4-5.1 [28.5 kB] Get:4 http://tw.archive.ubuntu.com/ubuntu/ oneiric/universe libjs-mochikit all 1.4.2-3fakesync1 [376 kB] ... Fetched 5,864 kB in 2s (2,346 kB/s) Preconfiguring packages ... Selecting previously deselected package wwwconfig-common. (Reading database ... 64249 files and directories currently installed.) Unpacking wwwconfig-common (from .../wwwconfig-common_0.2.2_all.deb) ... Selecting previously deselected package javascript-common. Unpacking javascript-common (from .../javascript-common_8_all.deb) ... Selecting previously deselected package libdbi1. Unpacking libdbi1 (from .../libdbi1_0.8.4-5.1_amd64.deb) ... Selecting previously deselected package libjs-mochikit. Unpacking libjs-mochikit (from .../libjs-mochikit_1.4.2-3fakesync1_all.deb) ... Selecting previously deselected package librrd4. Unpacking librrd4 (from .../librrd4_1.4.3-3.1ubuntu2_amd64.deb) ... Selecting previously deselected package ntop-data. Unpacking ntop-data (from .../ntop-data_3%3a4.0.3+dfsg1-3build1_all.deb) ... Selecting previously deselected package python-markupsafe. Unpacking python-markupsafe (from .../python-markupsafe_0.12-2build1_amd64.deb) ... Selecting previously deselected package python-mako. Unpacking python-mako (from .../python-mako_0.4.1-2_all.deb) ... Selecting previously deselected package ntop. Unpacking ntop (from .../ntop_3%3a4.0.3+dfsg1-3build1_amd64.deb) ... Selecting previously deselected package ttf-dejavu-extra. Unpacking ttf-dejavu-extra (from .../ttf-dejavu-extra_2.33-1ubuntu1_amd64.deb) ... Selecting previously deselected package ttf-dejavu. Unpacking ttf-dejavu (from .../ttf-dejavu_2.33-1ubuntu1_amd64.deb) ... Processing triggers for man-db ... Processing triggers for ureadahead ... ureadahead will be reprofiled on next reboot Processing triggers for fontconfig ... Setting up wwwconfig-common (0.2.2) ... Setting up javascript-common (8) ... Setting up libdbi1 (0.8.4-5.1) ... Setting up libjs-mochikit (1.4.2-3fakesync1) ... Setting up librrd4 (1.4.3-3.1ubuntu2) ... Setting up ntop-data (3:4.0.3+dfsg1-3build1) ... Setting up python-markupsafe (0.12-2build1) ... Setting up python-mako (0.4.1-2) ... Setting up ntop (3:4.0.3+dfsg1-3build1) ... Adding system user: ntop. Warning: The home dir /var/lib/ntop you specified already exists. Adding system user `ntop' (UID 106) ... Adding new group `ntop' (GID 114) ... Adding new user `ntop' (UID 106) with group `ntop' ... The home directory `/var/lib/ntop' already exists. Not copying from `/etc/skel'. adduser: Warning: The home directory `/var/lib/ntop' does not belong to the user you are currently creating. Wed Feb 15 14:31:59 2012 NOTE: Interface merge enabled by default Wed Feb 15 14:31:59 2012 Initializing gdbm databases Wed Feb 15 14:31:59 2012 Setting administrator password... Wed Feb 15 14:31:59 2012 Admin user password has been set Wed Feb 15 14:31:59 2012 Admin password set... Starting network top daemon: Wed Feb 15 14:32:00 2012 NOTE: Interface merge enabled by default Wed Feb 15 14:32:00 2012 Initializing gdbm databases ntop Setting up ttf-dejavu-extra (2.33-1ubuntu1) ... Setting up ttf-dejavu (2.33-1ubuntu1) ... Processing triggers for libc-bin ... ldconfig deferred processing now taking place
root@wistor-007:/var/lib/ntop$ cat /var/lib/ntop/init.cfg USER="ntop" INTERFACES="eth0,eth1" root@wistor-006:~$ sudo /etc/init.d/ntop restart Stopping network top daemon: ntop Starting network top daemon: Wed Feb 15 14:39:35 2012 NOTE: Interface merge enabled by default Wed Feb 15 14:39:35 2012 Initializing gdbm databases ntop
pjack@ubuntu:~$ ntpdc -p
remote local st poll reach delay offset disp
=======================================================================
=europium.canoni 192.168.0.3 2 64 3 0.30498 0.007932 1.98438
=211.79.171.150 192.168.0.3 3 64 3 0.05931 0.040888 1.98438
=220-133-13-3.HI 192.168.0.3 2 64 3 0.02657 0.004869 1.98438
=59-124-196-84.H 192.168.0.3 2 64 3 0.02979 0.008247 1.98438
=tock.stdtime.go 192.168.0.3 2 64 3 0.01994 0.017888 1.98438
pjack@ubuntu:~$ ntpdc -l
client europium.canonical.com
client 211.79.171.150
client 220-133-13-3.HINET-IP.hinet.net
client 59-124-196-84.HINET-IP.hinet.net
client tock.stdtime.gov.tw
pjack@ubuntu:~$ ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
59-124-196-83.H 59.124.196.87 2 u 16 64 7 30.366 4.155 2.271
220-133-13-3.HI 192.43.244.18 2 u 14 64 7 26.719 2.564 2.793
tock.stdtime.go 59.124.196.87 2 u 13 64 7 17.852 6.259 1.371
59-124-196-84.H 59.124.196.87 2 u 14 64 7 58.868 -17.627 10.498
europium.canoni 193.79.237.14 2 u 11 64 7 302.951 -2.471 2.228
pjack@ubuntu:~$ ntpq -pn
remote refid st t when poll reach delay offset jitter
==============================================================================
59.124.196.83 59.124.196.87 2 u 37 64 3 30.188 2.385 0.910
220.133.13.3 192.43.244.18 2 u 36 64 3 26.420 0.641 1.528
220.130.158.71 59.124.196.87 2 u 34 64 3 21.191 6.301 1.981
59.124.196.84 59.124.196.87 2 u 35 64 3 72.039 -25.230 20.354
91.189.94.4 193.79.237.14 2 u 33 64 3 302.555 -3.432 2.041
pjack@ubuntu:~$ more /etc/ntp.conf
# Specify one or more NTP servers.
server 0.ubuntu.pool.ntp.org
server 1.ubuntu.pool.ntp.org
server 2.ubuntu.pool.ntp.org
server 3.ubuntu.pool.ntp.org
# Use Ubuntu's ntp server as a fallback.
server ntp.ubuntu.com
# 加入要同步的 ntp server
# 加入本機, 並且設定成第十層(最大是 15)
server 127.127.1.0
fudge 127.127.1.0 stratum 10
# Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for
# details. The web page < http://support.ntp.org/bin/view/Support/AccessRestrictions>
# 預設先把所有的東西都關起來, 也可以寫成
# restrict default ignore
restrict -4 default kod notrap nomodify nopeer noquery
restrict -6 default kod notrap nomodify nopeer noquery
# 放行在同一個區網的 clinet 但不能修改
restrict 192.168.100.0 mask 255.255.255.0 nomodify notrap
# 放行自己
# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1
pjack@ubuntu:~$ sudo netstat -tlunp | grep ntp
[sudo] password for pjack:
udp 0 0 192.168.0.3:123 0.0.0.0:* 1947/ntpd
udp 0 0 127.0.0.1:123 0.0.0.0:* 1947/ntpd
udp 0 0 0.0.0.0:123 0.0.0.0:* 1947/ntpd
udp6 0 0 ::1:123 :::* 1947/ntpd
udp6 0 0 fe80::20c:29ff:fe78:123 :::* 1947/ntpd
udp6 0 0 :::123 :::* 1947/ntpd
pjack@ubuntu:~$ ntptime
ntp_gettime() returns code 0 (OK)
time d2da6bc3.34d94000 Mon, Feb 6 2012 7:09:55.206, (.206440),
maximum error 239875 us, estimated error 6508 us, TAI offset 0
ntp_adjtime() returns code 0 (OK)
modes 0x0 (),
offset 0.000 us, frequency 18.986 ppm, interval 1 s,
maximum error 239875 us, estimated error 6508 us,
status 0x1 (PLL),
time constant 6, precision 1.000 us, tolerance 500 ppm,
pjack@ubuntu:~$ ntptrace
localhost: stratum 3, offset 0.000000, synch distance 0.017913
220.130.158.71: timed out, nothing received
***Request timed out
pjack@ubuntu:~$ sudo service ntp stop * Stopping NTP server ntpd pjack@ubuntu:~$ sudo ntpdate tick.stdtime.gov.tw 6 Feb 07:28:08 ntpdate[2132]: adjust time server 59.124.196.83 offset -0.001831 sec
Function
|
Description
|
Performance
|
Storage arrays are required to quickly process a multitude of data I/O requests arriving simultaneously from hundreds (or thousands) of desktops and servers
|
Resiliency
|
Ø Automated RAID Rebuilds
Ø Data Integrity Checks
Ø “Phone Home” Alerts
Ø Environmental Monitoring
Ø Non-disruptive Software Updates
|
Virtualization
|
Ø Transparent Volume/LUN resizing
Ø Thin Provisioning
Ø Data Cloning
Ø Data Compression
Ø Data Deduplication
|
Human
|
PC
|
Sensory memory
|
buffers and registers
|
Short-term memory
|
cache, Random Access Memory (RAM), flash and solid-state disks (SSDs)
|
Long-term memory
|
hard disk drives
|
Era
|
Memory Type
|
40s-50s
|
Cathode Ray Tubes
|
50s-60s
|
Magnetic Core Memory
|
70s-present
|
Random Access Memory (RAM) integrated circuits
|
NAS Communications Protocols
|
|
NFS
|
NFS stands for the Network File System
protocol used by UNIX-
based clients or servers
|
CIFS
|
CIFS stands for Common Internet File
System. CIFS is another
network-based protocol used for file
access communications
between Microsoft Windows clients and
servers.
|
SAN Communications Protocols
|
|
Fibre Channel Protocol (FCP)
|
Typically occurs via specialized Fibre
Channel cabling, Fibre Channel host bus adapters (HBAs) and Fibre Channel
switches operating between the SAN and its hosts
|
iSCSI
|
iSCSI stands for Internet Small Computer
Systems Interface. iSCSI is “a transport protocol that provides for the SCSI
protocol to
be carried over a TCP-based IP network.”
|
FcoE
|
FcoE stands for Fibre Channel over
Ethernet. FcoE allows Fibre Channel storage traffic to be sent over Ethernet networks
|
mkcephfs -a -c /etc/ceph/ceph.conf --mkbtrfs
root@wistor-dev-7:~$ mke2fs -t ext4 /dev/mapper/ubuntu64--33--7-lvol0
mke2fs 1.41.14 (22-Dec-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
13107200 inodes, 52428800 blocks
2621440 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
1600 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 28 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
root@wistor-dev-7:~$ tune2fs -o journal_data_writeback /dev/mapper/ubuntu64--33--7-lvol0
tune2fs 1.41.14 (22-Dec-2010)
root@wistor-dev-7:~$ tune2fs -O ^has_journal /dev/mapper/ubuntu64--33--7-lvol0
tune2fs 1.41.14 (22-Dec-2010)
root@wistor-dev-7:~$ e2fsck -f /dev/mapper/ubuntu64--33--7-lvol0
e2fsck 1.41.14 (22-Dec-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mapper/ubuntu64--33--7-lvol0: 11/13107200 files (0.0% non-contiguous), 837781/52428800 blocks
root@wistor-dev-7:~$ cat /etc/fstab proc /proc proc nodev,noexec,nosuid 0 0 /dev/mapper/ubuntu64--33--7-root / ext4 errors=remount-ro 0 1 # /boot was on /dev/sda1 during installation UUID=bf0a72da-7ca7-4960-9a7e-f90298b95609 /boot ext2 defaults 0 2 /dev/mapper/ubuntu64--33--7-swap_1 none swap sw 0 0 # 加入這一行 /dev/mapper/ubuntu64--33--7-lvol0 /srv/osd.2 ext4 errors=remount-ro,data=writeback,noatime,nodiratime,user_xattr 0 1 root@wistor-dev-7:~$ mount -a root@wistor-dev-7:~$ mount /dev/mapper/ubuntu64--33--7-lvol0 on /srv/osd.2 type ext4 (rw,noatime,nodiratime,errors=remount-ro,data=writeback,user_xattr)
[osd]
; This is where the btrfs volume will be mounted.
osd data = /srv/osd.$id
[osd.0]
host = wistor-dev-5
# 把原本指定 btrfs devs 的這行拿掉
# btrfs devs = /dev/mapper/ubuntu1104--64--5-lvol0
[osd.1]
host = wistor-dev-6
# 把原本指定 btrfs devs 的這行拿掉
# btrfs devs = /dev/mapper/wistor--dev--6-lvol0
[osd.2]
host = wistor-dev-7
# 把原本指定 btrfs devs 的這行拿掉
# btrfs devs = /dev/mapper/ubuntu64--33--7-lvol0
|
RAID Technique
|
Description
|
|
No Parity
(RAID 0)
|
Ø increase I/O performance by striping (or logically distributing) data across
several disk drives.
Ø offered no protection against failed disk drives
|
|
Mirroring
(RAID 1)
|
Ø data is mirrored onto a second set of disks
Ø exacts a high capacity penalty
|
|
Fixed Parity
(RAID 3 / RAID 4)
|
Ø both use parity calculations (sometimes known as checksum) to
perform error-checking
Ø recovery of missing data from failed drives
|
|
Striped Parity
(RAID 5)
|
Ø parity is striped (or logically distributed) across all disks in
the RAID set in an attempt to boost RAID read/write performance
|
|
Multiple Parity
(RAID 6)
|
Ø using multiple iterations of fixed or striped parity on a group of
drives, which allows for multiple drive failures without data loss.
|
|
Approach
|
Description
|
|
Data replication
|
Mirroring critical data to an alternate
location
|
|
Data backup
|
Restore data that may have been
accidentally deleted or earlier data version
|
|
Tape-Based Innovations:
|
|
|
Interleaving
|
Ø improved tape backup speeds by allowing backups to be written to
multiple tapes concurrently
Ø writing to several tapes in parallel
|
|
Synthetic Backup
|
Ø required just a single full backup and used an intermediate
database to track and map the location of the continuous incremental backups
performed to tape thereafter
|
|
Reclamation
|
Ø Also pioneered by Tivoli Storage Manager (TSM)
Ø the tape reclamation process solved a problem created by Synthetic
tape backups
|
|
Disk-Based Innovations
|
|
|
Disk Staging
|
Ø data stored on optical media that could be moved by a staging
manager to magnetic disk drives
Ø could be used to send the data to tape without affecting
production workloads
|
|
D2D2T
|
disk-to-disk-to-tape
|
|
D2D
|
disk-to-disk without tape
|