1. Join Now

    AVForums.com uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Help me build a terabyte media server

Discussion in 'Desktop & Laptop Computers Forum' started by Steve Bate, Oct 21, 2003.

  1. Steve Bate

    Steve Bate
    Active Member

    Joined:
    Nov 10, 2002
    Messages:
    677
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    21
    Location:
    Tarporley
    Ratings:
    +11
    I've played around with ripping my CD's and DVD's and am now happy with accessability and WAF, the next stage is to build a muther of a media server.

    I want redundancy so I guess that a Raid 5 setup would give me some security, the last thing I want to do is to re-rip everything once it's up. I've got about 400CD's that I want to rip losslessly with APE, about 300 DVD's ripped with DVDShrink without compression and just the film and no extras, plenty of space for digital photos and also I'm in the process of capturing all of my camcorder footage. All in all I reckon about 1Tb useable should be ok to start with.

    What would be an ideal and cost effective setup re mobo, processor, RAM, HD controller, HDisks etc. What would be the best server software?

    Thanks in advance

    Steve
     
  2. MartinImber

    MartinImber
    Active Member

    Joined:
    Apr 5, 2001
    Messages:
    3,851
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    71
    Location:
    Worcester
    Ratings:
    +21
    What about Netware 6 - it can support Terabytes of disk space and I think you can find it free as a demo 2 user version.
     
  3. Bursar

    Bursar
    Member

    Joined:
    Jun 29, 2003
    Messages:
    2,405
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    63
    Location:
    Cambridge
    Ratings:
    +216
    But if you're not familiar with it, it's going to be fiddly to get everything setup. IIRC more disk space on a Netware server normally means more RAM. The other low cost option is Linux, but again, you may have a bit of a learning curve if you're not familiar with it.

    I suspect the easiest option is to use Windows (2K or XP - whichever takes your fancy) and just RAID the disks normally.

    Hardware wise, anything will do. Something AMD based will keep costs down, and you won't really need cutting edge stuff either.

    A Gigabit network is going to be handy, so stock up on a gigabit capable swicth, cables and network cards.

    As for disks, Some of the 200GB or 250GB units would be your best bet.

    However all that is going to generate some noise and heat. You'll probably need to put it in a room other than your HC, and don't forget to budget for a PSU that's beefy enough to run all the drives.
     
  4. Steve Bate

    Steve Bate
    Active Member

    Joined:
    Nov 10, 2002
    Messages:
    677
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    21
    Location:
    Tarporley
    Ratings:
    +11
    Netware :eek: :eek: Thats a name from the past!! I can remember selling my first Netware File server, 30Mb hard disk and we thought they'd never fill it!!

    I'll have a look, although it's a learning curve I'm not sure I want to take at the moment.

    Thanks anyway Martin

    Steve
     
  5. Steve Bate

    Steve Bate
    Active Member

    Joined:
    Nov 10, 2002
    Messages:
    677
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    21
    Location:
    Tarporley
    Ratings:
    +11
    Thanks for confirming much of what I was already thinking, the noise/heat issue isn't a problem. It'll be with all my other kit in a "comms room".

    How practical are IDE RAID controllers? I guess SCSI is still too pricey for the home mrket. Am I right in thinking that in a RAID5 environment effectively 1 disk is for parity ie 3 disks data/parity is striped across each disk therefore you "lose" 1 disk worth of capacity, then this decreases the "lost" capacity as you increase disk numbers? ie 3x250Gb disks used useable capacity is 500Gb, 5x250Gb disks used useable is 1Tb?

    What happens if you add disks, does the RAID rebuild itself?

    Sorry for the questions
    Steve
     
  6. Mr.D

    Mr.D
    Well-known Member

    Joined:
    Jul 14, 2000
    Messages:
    11,040
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    133
    Ratings:
    +1,111
    We use IDE based raids on our workstations at work. They work great. Got a terrabyte on each machine whereas a couple of years ago we only had a couple of terrabytes for the entire system.

    We used to have to keep an eye on diskspace 20gb scsi for local disks was normal. I can do a years work these days without cleaning anything out and only fill about 20%. I sometimes have to double check when I check my diskspace as I was so used to seeing a value of 90% used rather than available!)

    They do tend to go bang a fair bit more often than the scsi but we have the valuable data backed up centrally for this eventuallity.

    Its noisy though we have the raids remotely in a machine room .
     
  7. Yandros

    Yandros
    Well-known Member

    Joined:
    Oct 13, 2003
    Messages:
    2,711
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    133
    Ratings:
    +531
    Well, if you want a REALLY serious AV server, I can recommend the machine we bought recently as a main office server.

    http://www.supermicro.com/PRODUCT/SUPERServer/SuperServer7043P-8R.htm

    has 10 hotswap drive bays, with 3 huge pull out fans behind the 2 drive cages. Mobo supports 2x3.2Ghz xeon CPUs and a huge amount of RAM, and is PCI-X. There are 3 200W hotswap power supplies, and 2 gigabit LAN ports. Main system cooling is a massive 120cm fan in the back. Only downside is that the USB ports are 1.1, so you might need to chuck in a USB2.0 card.

    The case is either floorstanding, or rackmountable.

    It is however EXTREMELY noisy! I cannot stress this enough. If you're running it in a comms room though it should be fine.

    We fitted ours with hardware RAID5 and 5x36GB SCSI drives, but I notice that our builders will fit 146Gb Maxtor Atlas 10K drives, so about 10 of those will give you just over 1TB I think in RAID 5 configuration. You might be able to dramatically lower the price by going for IDE drives, but I've no experience of using them with RAID5 personally.

    As others have mentioned, this number of drives is going to kick out a lot of heat, so even if you can't run to a server class motherboard etc, I would still make sure your disk subsystem has server standard cooling.

    For this sort of exotic AV kit I would recommend the guys who build both our office machines and my home gaming rigs

    www.armari.co.uk

    You can use the instant system pricer to get a custom quote, but I'd have a stiff drink ready...146GB SCSI drives ain't cheap!

    They specialise in bespoke high end graphics workstations etc, so should be able to give you some good advice if you give them a call.
     
  8. MartinImber

    MartinImber
    Active Member

    Joined:
    Apr 5, 2001
    Messages:
    3,851
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    71
    Location:
    Worcester
    Ratings:
    +21
    We replaced a Win2000server with a Netware 6 server earlier this year.

    Stonkingly fast with Advantage Database Server.:eek: :eek: :eek:

    My boss has installed two so far this month for upgrades
     
  9. JohnS

    JohnS
    Active Member

    Joined:
    Oct 31, 2000
    Messages:
    2,580
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    71
    Location:
    Herne Bay, Kent
    Ratings:
    +33
    Hi Steve,

    I've recently been considering the exact same thing as the missus is moaning about the space taken up with cd and dvd racks. I already have a server running w2k server, stuffed in the understairs cupboard, no monitor, keyboard or mouse, I just remote desk into it if I need to change anything and by using W2Kserver over XPpro allows more than one terminal services connection at the same time if that might be important. I also think theres a limit to concurrent connections to a shared drive using xp.

    I've been looking at hardware raid boards as the next upgrade. The most commonly talked about appear to be the 3ware ones but they seem to be quite hard to come by here but I recently exchanged emails from someone who sells/distributes them in the uk:-

    The compant is www.tmc-uk.com and the lady I spoke with was Eliza Bogan, Sales Executive
    Tel :01438 842304
    Fax :01438 842308

    They tell me they can be live expanded, that is with the machine switched off an extra drive can be added to expand the raid. prices from an email:-

    Parallel:
    7506-4 £171
    7506-8 £275
    7506-12 £375

    Serial:
    8506-4 £221
    8506-8 £339
    8506-12 £485

    These cards (although iut may be windows, I'm not sure which) all have a 2TB limit.

    Looking around at the moment, ebuyer seem to have the best value drives being 200gb maxtor oems here

    Linux allows you to create a raid 5 array in software that can be live expanded, windows 2kserver does not. I'm a bit concerned of the learning curve involved with Linux as I've never touched it.

    As far as most of the hardware is concerned I dont think it matters all that much as long as you have 1/2gb+ of ram and a fast enough network.

    Some food for thought.
     
  10. Badger

    Badger
    Standard Member

    Joined:
    Jan 21, 2002
    Messages:
    257
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    18
    Ratings:
    +1
  11. tomson

    tomson
    Active Member

    Joined:
    Jul 19, 2000
    Messages:
    1,918
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    66
    Location:
    Berk'amsted
    Ratings:
    +187
    I know a few hardened PC guys who now swear by Xserve RAID set ups. Probably not quite what you're after though.
     
  12. Badger

    Badger
    Standard Member

    Joined:
    Jan 21, 2002
    Messages:
    257
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    18
    Ratings:
    +1
    This is correct - the system will also continue to work even if one drive fails - and that drive rebuilt when a new one is inserted. 2 drives fail together and it's all gone.

    Phil
     
  13. Steve Bate

    Steve Bate
    Active Member

    Joined:
    Nov 10, 2002
    Messages:
    677
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    21
    Location:
    Tarporley
    Ratings:
    +11
    I've seen NAS for £1k for 500Gb, over £2.3K for the Dell soln is a little steep.

    The other alternative is external Firewire drives at £200 for 250Gb, I could daisy chain 4 x 250gb giving me a terrabyte for £800, granted it doesn't give me redundancy but it's a darn site cheaper than NAS.

    The more I think about it, £800 for 1Tb today, if something fails I've only got he worry of 1x250gb drive to recut and the price of the storage is continuing to fall.

    More suggestions? It makes a more informed decision easier :p

    Steve
     
  14. Steve Bate

    Steve Bate
    Active Member

    Joined:
    Nov 10, 2002
    Messages:
    677
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    21
    Location:
    Tarporley
    Ratings:
    +11
    2 drives fail at once :eek: There's only so far you can take redundancy. Terabyte storage and terabyte backup....following redundancy, I know corporates who baulk at this :eek: :eek: :D

    Steve
     
  15. m@rk

    m@rk
    Well-known Member

    Joined:
    Jan 3, 2002
    Messages:
    1,212
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    83
    Location:
    Lincolnshire
    Ratings:
    +360
    Part of my job I looking after our content delivery network at work. To give you an idea of scale, this is the one that MSN used to do the live broadcast of the Kylie concert in Manchester last year where we had 300,000 concurrent 300k streams.

    We are able to convert live content in real time to windows media, real and quicktime and deliver the above.

    Now this sort of this is overboard for what you want BUT as you can guess, we have some pretty serious media servers.

    If you want to do this properly with media being served via play lists over the network as opposed to just opening a file off a shared hard drive, look at using windows media. Sitting on Win2K (or Windows 2003), it will do everything that you want from both a streaming standpoint and storage.

    Unlike Netware, the learning curve is not as steep as it's the same old windows GUI.

    On the other hand, if you just want storage, then any bunch of disks will do running on say Win2K pro or XP although look at having your storage in a separate box.

    As a minimum I would say RAID 5 on your disks but this does not offer vast redundancy and there is a performance hit on all but the high end systems. Having said that though, as you don't have loads of users, a low end system with low throughput may well do the job okay. The spec of the PC itself will not really change regardless of the storage capacity so spec the PC for delivering the media as if it had a 1-GB drive (so the normal HCPC kind of thing).

    If however you plan on playing the actual files on another machine, then the spec of the 'server' can be pretty low (Celeron CPU, 256-MB RAM etc) as it will only need to serve files.

    My own Windows 2003 server sat next to me is a Celeron 433MHz with 512-MB of RAM. Is runs like a dream because it isn't doing anything except service files to other PCs. In my case, I have a small Compaq storage array connected via fibre to it with 10 x 72-GB drives in it. All the clever stuff is done by the storage array and not the server hence a low spec box.

    So don't get carried away just because you can see big storage numbers.

    Have fun
     
  16. halephil

    halephil
    Standard Member

    Joined:
    May 24, 2002
    Messages:
    61
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    8
    Ratings:
    +1
    I’m currently testing such a ‘media’ server using a 3ware 8506-8 with 8x250GB drives configured as Raid 5 (effective storage is 1630GB) in a www.tmc-uk.co.uk distributed RM311 case. Pricing works out at around £1/GB of Raid 5 storage plus the cost of the host PC system which, as has been pointed out earlier, doesn’t necessarily have to be all that special for this type of application. The key things to consider are the power requirements and cooling for which the server cases (approved by Intel and 3Ware) are best – although not that cheap.

    You can assign a drive as a ‘hot spare’ which the system will use to rebuild the array in the event that a drive ‘degrades’ however this ties up one of the controller card ports. The more likely setup is to have a ‘cold’ spare drive and use this as soon as the system alerts you that a drive has failed running the risk that you don’t have a second disk fail whilst the array is not fault tolerant.

    If anyone’s interested in more information on how it all performs drop me a line.

    Cheers, Phil
     
  17. Branxx

    Branxx
    Standard Member

    Joined:
    Nov 1, 2000
    Messages:
    801
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    18
    Location:
    London, UK
    Ratings:
    +1
    Steve,

    I have been operating a media server of the type you are describing a while ago. I am also currently in the process of upgrading the box so I can share my component choices.

    The old server is running an Asus P2B-D dual Pentium 3 motherboard with two 500MHz processors and 1GB memory. I am using a simple Promise ULTRA 100 IDE RAID controller in JBOD mode. Together with the IDE on the mother board the server has 6 Maxtor data disks (4 x 200GB, 2 x 120GB), Primary IDE runs the system disk and CD-ROM drive.

    The system is in Lian-Li PC-61 black aluminium case that has plenty of space and well positioned fans.

    It has been switched on without break for the last 4 years, running Windows 2000 Advance Server. I have successfully tested this configuration with 3 concurrent PowerDVD sessions. Playing audio file over the network is much less demanding.

    The server is also running DHCP, DNS and routing services. It also acts as network boot server for diskless client PCs around the house (I have two of them).

    The new server is based on Asus PC-DL Deluxe dual processor motherboard with two XEON 2.8GHz and 2x512MB Corsair Twinx 3200XLL memory. The motherboard has simple RAID controller that can support four SATA drives and two IDE drives. That is in addition to two standard IDE channels.

    Again, I opted for Lian-Li PC-61 case, but this time I plan to use 4 Maxtor 250GB SATA drives and 2 300GB drives, making a total of 1.6TB. I haven’t decided yet if this will be running as JBOD or RAID 0. The operating system is probably going to be Windows Server 2003 for Small Business.

    I am not concerned with redundancy because a) I already have the backup for all the media data (i.e. the original CDs and DVDs); b) I am planning to run a synchronisation software between my server and my friends server (over VPN) so that all my data is replicated on his machine and all of his data is on my machine. This will in effect create not only double redundancy but also combine the two media libraries into one large. This obviously requires an ADSL always on connection with the static IP and VPN router on both ends.
     
  18. Steve Bate

    Steve Bate
    Active Member

    Joined:
    Nov 10, 2002
    Messages:
    677
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    21
    Location:
    Tarporley
    Ratings:
    +11
    Branxx,

    You never cease to amaze me :eek: :D :eek:

    I am still considering the JBOD over RAID, the issue is the time it takes to rerip everything, I also own all the disks, how much cost do you associate with the time it takes to rip all your DVD's & CD's.

    Steve
     
  19. JohnS

    JohnS
    Active Member

    Joined:
    Oct 31, 2000
    Messages:
    2,580
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    71
    Location:
    Herne Bay, Kent
    Ratings:
    +33
    Far less than the cost of a raid controller card and 1 extra drive!

    Anyone know what benefits choosing W2k3 server over W2K server will have, I know its supposed to be a slightly faster OS but will we actually notice it when all your doing is connecting to shares?
     
  20. Branxx

    Branxx
    Standard Member

    Joined:
    Nov 1, 2000
    Messages:
    801
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    18
    Location:
    London, UK
    Ratings:
    +1
    “How much cost do you associate with the time it takes to rip all your DVD's & CD's?”

    I friend of mine has commissioned his teenage son to do all the ripping (some 350 disks). He pays him an equivalent of minimum wage per hour that turns out to be approx £0.60 per disk (so junior ‘earns’ approx. £200 for doing something useful).

    JBOD has advantage over simple RAID 0. In the case of failure, you’ll only loose one disk as opposed to everything on the RADI 0. If finance permits, than RAID 5 is certainly an elegant solution.

    I would generally recommend using slower 5400 RPM disks as they are quieter, running less hot, and are likely to last longer. They are still too fast for any media applications.
     
  21. m@rk

    m@rk
    Well-known Member

    Joined:
    Jan 3, 2002
    Messages:
    1,212
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    83
    Location:
    Lincolnshire
    Ratings:
    +360
    I can probably think of a lot of reasons why not to change (it's still new and awaiting it's first service pack) but as a file server it works very well.

    It does have a lot more disk tools in it (which is handy for what you are doing) and does run well on my low spec server (see my earlier comments).

    I have had it running since launch and have not had any problems with it although it's only a file server so it isn't doing much.
     
  22. Steve Bate

    Steve Bate
    Active Member

    Joined:
    Nov 10, 2002
    Messages:
    677
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    21
    Location:
    Tarporley
    Ratings:
    +11
    The firewire external drives are looking more promising now then... 1Tb for £800 with the obvious proviso that if a drive fails you've got a lot of re-ripping to do, I could also hang them off the back of my existing (small) media server. Any thoughts on how well they'd work streaming 2 x DVD and 1 zone of audio across switched 100mb or do I really need to put in gigabit Ethernet.

    I'm also playing with the recent 54g wireless, still waiting for the AP so no idea what streaming DVD will b elike over wireless.

    Steve
     
  23. m@rk

    m@rk
    Well-known Member

    Joined:
    Jan 3, 2002
    Messages:
    1,212
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    83
    Location:
    Lincolnshire
    Ratings:
    +360
    Depending on how you do things, your DVD quality stream should NEVER be more than 2mb.

    Considering that you have a 100mb network, I would say no problem as long as your network is not full of other high bandwidth stuff.
     
  24. dannyc

    dannyc
    Guest

    Products Owned:
    0
    Products Wanted:
    0
    Ratings:
    +0
    Steve,

    I've been thinking along the same lines about firewire drives. Having already got all my music on a Lacie 500gb firewire big disk, I just got 2 of the new Maxtor 250gb firewire drives for £186 each inc VAT from ebuyer. They are very quiet and come with retrospect backup software allowing easy disk to disk, tape, or cd/dvd archiving. I was thinking of making complete copies of my hard drives on extra firewire drives and taking them out of the house completely in case of theft/fire.

    Actually, coming to think of it, any room at your house for a couple of disks Branco?

    Dan
     
  25. Branxx

    Branxx
    Standard Member

    Joined:
    Nov 1, 2000
    Messages:
    801
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    18
    Location:
    London, UK
    Ratings:
    +1
    Dan,

    There is a plenty of space in my cellar and if you bring them personally I'll take Helen and you for dinner.
     
  26. Steve Bate

    Steve Bate
    Active Member

    Joined:
    Nov 10, 2002
    Messages:
    677
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    21
    Location:
    Tarporley
    Ratings:
    +11
    Muhaahahahaha

    I'll get me coat :rolleyes:

    Steve
     
  27. Madders

    Madders
    Active Member

    Joined:
    Jan 30, 2002
    Messages:
    2,303
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    51
    Location:
    London
    Ratings:
    +49
    What a great thread, I've been toying with a similar idea!

    Dannyc - Forgive me for sounding stupid, but the Lacie big disk, do you just play your stored songs straight from that drive? Or do you just use it as a backup? If you can play songs/movies from it, is there any loss in quality via firewire, or any pausing whilst info is being accessed?

    Cheers,

    Steve
     
  28. m@rk

    m@rk
    Well-known Member

    Joined:
    Jan 3, 2002
    Messages:
    1,212
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    83
    Location:
    Lincolnshire
    Ratings:
    +360
    Isn't that exactly what Fred West said? [​IMG]
     
  29. Steve Bate

    Steve Bate
    Active Member

    Joined:
    Nov 10, 2002
    Messages:
    677
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    21
    Location:
    Tarporley
    Ratings:
    +11
    Steve,

    When I started this thread I said that I had been playing with ripping DVD and CD and this revolved around playing with an external maxtor firewire drive, with this hanging off a small server. I can now confirm that there is no reduction in quality at all either audio or video. It was this that made my mind up o do something, either JBOD or a proper RAID Media Server.

    Steve (Still thinking hard but swaying towards firewire)
     
  30. kryten

    kryten
    Active Member

    Joined:
    Apr 9, 2003
    Messages:
    515
    Products Owned:
    0
    Products Wanted:
    0
    Trophy Points:
    18
    Location:
    Swindon
    Ratings:
    +14
    I am using a 3ware 8500-4 Serial ATA controller with four 160Gb discs, giving me 480Gb of RAID5 storage.

    With the 3ware arrays you CANNOT add other discs later to extend the array (or replace the discs with bigger ones to increase it) - you need to destroy and re-create the array to do that.

    Serial ATA makes the cabling a lot easier inside the case which also allows for improved airflow (normal IDE cables really play havoc with front fans).

    I'm using the whole thing in an Antec Sonata case 120mm fans and a Silent CPU cooler. Aside from low level disk access noise it is pretty much silent.

    Excellent solution, in my opinion and very fast!
     

Share This Page

Loading...