It's been several months since the original question, but it took a bit
of reconfiguration and testing to get good results. Last night the
incoming Sprintlink feed was about 10gb and 5.7% of the articles were
dropped. The total articles offered by Sprint and ANS combined were 1.2M.
1. 4/200 processors were upgraded to 5/250
2. Memory was upgraded from 256mb to 512mb
3. LSM 2xKZPAA/7xRZ29 stripe set was replaced with KZPSA/HSZ50+64mb/
12xRZ28D striped for non-binaries, 4xRZ40 striped for binaries.
4. Perl filtering (cleanfeed) was enabled to reduce spam - this works well
5. NNmaster was eliminated and replaced by overview
6. The T1-only feed was changed to one T1 and second over a T3
(the network interface is a DEFPA on the same FDDI ring as the router)
7. Joe Greco's timer patch was installed to measure the effect of changes
8. Several new innds were installed (now 1.7.2)
9. New dbz was installed (3.2)
The process was iterative, the better the system worked, the more
articles were received, etc.
The below e-mail from Marco Luchini was helpful. Ann Cantelow performed
endless compiling and testing.
expire.ctl:
*:A:0:0:0
*:U:1:10:10
*:M:1:10:10
alt.binaries.*:A:1:2:2
alt.binaries.warez.*:A:0:0:0
control.*:A:1:1:1
Filesystem 512-blocks Used Available Capacity Mounted on
/dev/rz17c 47790112 17906830 25104270 42% /usr/var/spool/news
/dev/rz19c 68951594 40235300 21821134 65% /usr/var/spool/news/al
t/binaries
timer output on five minute intervals (300,000 ms):
Apr 19 08:53:35 apollo innd: ME time 300001 idle 88435(11650)
artwrite 64455(1424) artlink 6(585) hiswrite 26526(2946) hissync 151(377)
sitesend 184(2848) artctrl 13540(315) artcncl 0(0) hishave 31227(3374)
hisgrep 647(305)
OR 70% busy (of a processor) (300000-88435)/300000
45ms article write 64455/1424 (LSM was 500+ ms)
9ms history write 26526/2946
9ms history lookup 31227/3374 <<< needs work
Usenet is not as easy as it used to be.
John Nebel
---------- Forwarded message ----------
Date: Wed, 31 Dec 1997 12:39:08 GMT
From: Dr Marco Luchini <m.luchini_at_ic.ac.uk>
To: John Nebel <nebel_at_athena.csdco.com>
Subject: Re: Summary: I/O tuning on LSM Usenet spool (and further question)
John Nebel writes:
>
> Received the below reply from Alan Rollow. Thanks. He says, and
> observations show this too, that large average write times due to periodic
> syncs of a dirty cache don't necessarily slow down reads.
>
> OPERATIONS BLOCKS AVG TIME(ms)
> TYP NAME READ WRITE READ WRITE READ WRITE
> vol newspool 31141226 28051270 471444624 455973046 18.9 516.9
>
> However, the i/o system isn't fast enough to keep up with the news feed.
> About 2/3 of the feed is being dropped. It's true that the design of innd
> is not great with the way it relies on the file system as a database
> manager. There are lots of little writes all over the disk.
>
> What would be faster than the existing 2 x KZPAA + integrated controller and
> 7 x RZ29 stripe set? At least 2x i/o improvement is needed.
>
> Would KZPSA + RZ29W stripe set, or HSZ50 with a big cache + RZ29 array, or
> KZPAC + RZ29W show significant gains? Is there another, faster
> combination?
HSZ50 (or HSZ70 now) plus ultra wide disks would definitely be faster,
though you don't get any real bennefit on small transfer from ultra.
LSM puts quite a lot in between the disks and the application and I've
never felt it's particularly fast. Hardware striping is much better but
a lot more expensive, typically.
KZPAC might well be enough if you don't need oodles of disk space.
You might want to play with the I/O parameters a bit before changing the
hardware though. Decreasing ubcmaxpercent should force data to be
written more frequently and therefore decrease the amount of jumping
around the stripe sets that you have to do.
It may well be a case that the old rule about the number of spindles is
what you have to go on. You will get twice as many I/O's per sec from
14 RZ28s than from 7 RZ29s. And if it's I/Os per sec that you are after
rather than MB/s then using faster disks will not help that much. It's
much better to use lots of them.
Ciao, Marco
Received on Mon Apr 20 1998 - 22:57:47 NZST