Home NAS suggestions

[amenditman]

I just got a package from Newegg I've been waiting a long time for.

Not that they took a long time to ship, but that it took me a long time to pull the trigger and send them the PayPal payments.

Waiting patiently paid off, I was able to grab these items individually over a three month period for ~ 65% of retail.

Still, it was ~$600.

 

Here's what it contained.

 

4 @ Western Digital Red WD30EFRX 3TB IntelliPower 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive -Bare Drive

IntelliPower 64MB Cache

SATA 6.0Gb/s

For NAS systems

http://www.newegg.co...N82E16822236344

 

1 @ Sans Digital 4 Bay eSATA Port Multiplier JBOD Tower Storage Enclosure TR4M+BNC

SATA 150 & 300

4 x Hot-Swappable 3.5" Drive Bays

JBOD (Hard drives are accessed individually)

eSATA (via Port Multiplier)

OS Supported Supports PC, Linux and MAC OS, including Windows 7 and Snow Leopard

http://www.newegg.co...N82E16816111177

 

1 @ HighPoint RocketHybrid 1222 PCI-Express 2.0 x1 Low Profile Ready SATA III (6.0Gb/s) Controller Card

2 x eSATA III External Connectors

Up to 6Gb/s

http://www.newegg.co...N82E16816115098

 

I want to set this up as a home file storage/backup server. Probably attached to my main desktop running ArchLinux.

No Windows sharing is necessary, but I would like to be able to access it from outside the network from my ArchLinux laptop or Android tablet.

 

What advice do you have? What RAID levels would you suggest? What filesystem? What should I read about?

I've been reading like crazy about a lot of things, but.....choices, choices. WOW!

 

Thanks for your suggestions.

Edited February 13, 2013 by amenditman

[V.T. Eric Layton]

10 different folks are going to give you ten different suggestions here.

 

For my part, I'll tell you that I'm very rusty on RAID. I do know that software RAID used to be very poorly supported in Linux. I don't know about these days. For a server application, I'd probably run JBOD, unless you want some kind of redundancy/backup of the data; in which case you should mirror your drives. If those 3TB drives are identical, mirror 1-2, and 3-4.

 

Set up a small separate drive for the OS (not part of the RAID array).

 

Lastly, I love Arch Linux. It is a wonderful distribution of GNU/Linux, however, as we discussed the other night via phone, a rolling-release distribution is not what I would use for a server application... not stable enough, in my opinion. You're not going to be surfing the net and making PowerPoint presentations with this OS. You just want something simple and ROCK-ARSE SOLID. Can you think of a distro that fits that description? Yes, I thought you could...

 

 

 

To access from a machine other than your main system, you should connect the server to the one of the switchports on your ISP-provided equipment. Assign it a static IP and you'll be able to ssh or ftp into it from any device on your home network. You cannot access it from outside of your SOHO (small office or home) network unless your ISP is going to provide you with a static IP address... at least I don't think so. Or maybe I'm misunderstanding what you're trying to do here.

 

Clarify, please.

 

I'll check this thread tomorrow. Gotta' go study now.

 

P.S. Pretty good article on Wikipedia, which I'm sure you've already seen --> https://en.wikipedia...ttached_storage

 

P.P.S. From SlackDocs Wiki:

 

JFS

 

 

JFS was contributed to the Linux kernel by IBM and is well known for its responsiveness even under extreme conditions. It can span colossal volumes making it particularly well-suited for Network Attached Storage (NAS) devices. JFS's long history and thorough testing make it one of the most reliable journaling filesystems available for Linux.

[amenditman]

The High Point Hybrid Rocket card was a bundled item. I was not planning on using it as I will not be using an SSD with this.

 

I was originally thinking JBOD and just rsync'ing to my off-site hosting account for data preservation. But then I started really reading and there are sooooooo many choices I am not sure exactly what I want to do.

 

I could use the workshop computer as an always on server to attach this to. It runs Mepis and is rock stable (probably a better choice than my Arch workhorse) and boring. Just what you want in a computer that needs to just do it's job and always be available. The enclosure itself is just that, an enclosure for storage devices. Not an NAS that requires a separate OS.

 

I will just be storing my projects, school files, family photos, home movies, and backups of the various computing devices I have running around here. Possibly the same for my 4 siblings and parents (and maybe a few select friends) if I can have fairly secure access from outside my SOHO network.

 

The files on this storage device will be backed up to my off-site hosting account weekly. Nothing too fancy, just rsync in the middle of the night.

Edited February 13, 2013 by amenditman

[ichase]

I'm running a basic Arch setup (no gui) on a headless server with 2 TB of storage space. I am sharing to Windows computers as well so I am running SAMBA. I secure shell into my server to run updates. You would be good to go with the box running mepis to host your server. I really need to learn this rsync thing, sounds like a great idea along with piece of mind knowing your data is save somewhere else in case of an Oh Shizzle.

[V.T. Eric Layton]

How do you plan to access the NAS from outside of the local network where it resides? Your dad could not access it from his home using his computer unless he knew your current IP address and you set your firewall to allow access from outside. Unfortunately, your ISP will occasionally reissue you a dynamic IP address. You'd have to alert everyone to the new IP each time it changes.

 

Uh... or is my brain just not functioning today? That's a possibility. I had a bad night... something I ate didn't like me. I'm still nauseous this morning. Staying home from class today because of it.

[securitybreach]

I am envious of your new purchase....... I need 12tb as my 5.5tb is getting a bit full.

[V.T. Eric Layton]

What the heck do you guys put on these drives? I'm still running 750Gig on my main system and I barely utilize 50% of the available space.

[securitybreach]

What the heck do you guys put on these drives? I'm still running 750Gig on my main system and I barely utilize 50% of the available space.

 

I would tell you but then I would have to kill you

[amenditman]

What the heck do you guys put on these drives? I'm still running 750Gig on my main system and I barely utilize 50% of the available space.

We don't talk about things like that in polite company!

[amenditman]

I was looking at RAID, and pretty much discounted RAID 5 (ArchWiki RAID article has links to resources why not to use). RAID 6 or 10 were what I was looking at.

I was never considering doing RAID in software. Hardware RAID controller only.

 

Also, configure each logical disk to less than 2tb, unless you like to work outside the bounds of most OSs.

Where can I read details about this and what I need to know to do it? And why?

 

But then I started really looking at LVM2 documentation and it might do what I want, without any RAID.

 

Here is what I am thinking.

Create 4 physical volumes, one on each physical disk (I can't keep much more that that straight in my mind)

Create 1 volume group of 4 stripes across the 4 physical volumes (speed up access to large files and minimize head travel?)

Create logical volumes for data access and organization (still reading about mapping lv's to pv's within the vg) :'(

Use snapshot feature and rsync to create regular backups, stored locally and off-site (don't have much info on this yet, but it's only reading a few whole bunch of how-to's) My data changes very slowly/infrequently, mostly just reads with occassional large additions. (?Is it possible to access the snapshot if it is stored on my hosting server?)

Does this make any sense, or have I got it all mixed up?

Edited February 13, 2013 by amenditman

[burninbush]

I gotta ask, what sort of home network can utilize the speed of a raid disk? Is there any point in having the server be faster internally than the network pipe can pass data?

[burninbush]

 

@burninbush - no, generally raid does =not= make a set of disks fast, rather, it slows them because the raid array must calculate the parity bits.

the only raid the speeds up the disk array (only under some circumstances!) is raid 0 where 2 disks share the work load. but loosing 1 hard drive causes all data to be lost. (there's no way to calculate what was on the other disk - there's no parity bit(s)).

 

 

OK, but that misses the point I wanted to inquire about; if a single drive can produce data in excess of 100 MB/second, then what is the reason to make a raid on a home machine? It just seems like the raid would spend a lot of time waiting for an opportunity to shoot out another packet. If it's just to store large amounts of data then a bunch of single drives would avoid all the raid hassle.

 

Aren't raid5 and 6 generally much faster than a single drive? But again, if you only have a 1gb network [or one much slower] then how could you use the bandwidth?

 

My NAS here is an aged HP laptop with a dead screen; it connects through the wifi at 54mpbs, which even the old 4200 rpm laptop disk can keep busy.

[V.T. Eric Layton]

He might want to create a RAID mirror of his drives, as I mention way up top here, in order to backup important data on his server.

[Peachy]

I'd go with a QNAP TS-469 and set it up as a RAID 5 array.

[amenditman]

I've been reading a lot. The correct title for this thread should probably have been "Home Network File suggestions". I never really considered setting up a separate computer as a file server.

 

In several posts here at BATL, securitybreach mentioned using NFS instead of SAMBA and that caught my interest. I can see how it would be easy to set up on top of LVM, just can't find anything to indicate whether or not it is a shrink/expand friendly file system like ext4 is. It's not crucial, but I'd like to know.

 

So the plan (such as it is) currently:

4 disks as 4 Physical Volumes

1 Volume Group

Logical Volumes as needed for organization and management - striped across the 4 physical volumes (one for family shared files, photos, videos, etc.; one for all my important stuff)

Format to either ext4 or nfs

Set up a cron job to create snapshots of LVs and make backups to local and remote storage with rsync

 

Anyone with real world experience setting up LVM striping? Sure would be nice to have a mentor on this.

Advice on using NFS on LVM?

[V.T. Eric Layton]

Sorry. If you were troubleshooting a small business network, I might be able to help. Otherwise, all previous knowledge has been pushed out of my head this past few weeks. My eyes are falling out. I can't even remember my name at the moment. I keep thinking it's Cisco, but that can't be right. Maybe Frisco? Crisco?

 

*sigh*

 

I have to go study now...

[securitybreach]

Lvm is the software you use on your file server to create logical volumes to hold the file systems you export with NFS. So, for example, use lvm to create the logical device /dev/md0 on your file server. On /dev/md0 then you create an ext4, btrfs, or whatever file system which you export to the clients via nfs.

 

OK so now you have a logical volume but on it you have to create a file system; i.e., ext4, xfs, btrfs, etc. Assuming that you have created the file system - it's this file system that you share with NFS - NOT the logical volume you created with lvm. Once you have the file system created you will typically place the mount point of that file system in /etc/exports for sharing to client systems.

http://www.linuxforu...p=8801#post8801

[amenditman]

Otherwise, all previous knowledge has been pushed out of my head this past few weeks. My eyes are falling out. I can't even remember my name at the moment. I keep thinking it's Cisco, but that can't be right. Maybe Frisco? Crisco?

 

*sigh*

 

I have to go study now...

Maybe Frito Bandito. I know what you mean, sometimes the cramming gets overwhelming.

Lvm is the software you use on your file server to create logical volumes to hold the file systems you export with NFS. So, for example, use lvm to create the logical device /dev/md0 on your file server. On /dev/md0 then you create an ext4, btrfs, or whatever file system which you export to the clients via nfs.

 

 

http://www.linuxforu...p=8801#post8801

So, even though it's called NFS, it's not really a filesystem like ext4 or btrfs. I need to format my LV's as ext4 and then make them available through NFS. That clarifies and simplifies things a bit. I understand ext4 on LV's.

 

Now I need to get my head around striping LVM. More reading, followed by more reading, and probably some more questions.

 

Thanks

[crp]

over at betanews , one of the columnists is talking about setting up his home network file server

(hint - )

[V.T. Eric Layton]

Ay yi yi yi... I am the Frito bandito...

 

http://youtu.be/FbYj7ZyqjYY

 

The Cisco Kid was a friend of mine

He drink whiskey, Pancho drink the wine

 

http://youtu.be/vYTX6DcAlBU

[crp]

omg, Eric has lost it.

[therock247uk]

lol

[sunrat]

omg, Eric has lost it.

La La Land.

[securitybreach]

omg, Eric has lost it.

 

Lost it? You mean he found it and lost it again?

[V.T. Eric Layton]

Found what? What? Or as my Brit mates like to say after a few pints, "Jolly good show! Wot? Wot?"

 

[amenditman]

omg, Eric has lost it.

I think Eric is lost IN it.

[V.T. Eric Layton]

Bob, would this be of any interest to you?

 

http://lifehacker.com/5986883/how-to-combine-multiple-hard-drives-into-one-volume-for-cheap-high+capacity-storage?utm_source=Lifehacker+Newsletter&utm_campaign=94d4f84041-UA-142218-1&utm_medium=email

[amenditman]

Bob, would this be of any interest to you?

 

http://lifehacker.co...tm_medium=email

That's a great LH article Eric. Most of it applies to Windows but a few bits for the Mac/Linux crew.

[amenditman]

I've looked at everything you all have suggested, whatever I could find and read (lots of pages on the ArchWiki and links from there), and discussed it with my Linux system administration professor.

 

My data is relatively unimportant and changes slowly.

 

Here is what I am going to do.

I will have a weekly cron job to make local and offsite backups. Anything important gets the instant backups treatment.

The Sans Digital tower has a SATA port multiplier built into it (the SATA specification allows for as many as 15 devices to be plugged into a single port with the use of a port multiplier, very similar to SCSI) and I will be able to access all 4 disks over a single eSATA cable. They will share bandwidth on the SATA port which is a total of 3.0 Gbps, average HDD read/write speed is only in the 40 - 60 MBps range. 60 MB is less than 500,000 bits, (please check my math, I think it's correct but not sure) so I could run 6 of these drives before I overwhelmed the 3.0 Gb bandwidth.

I am going to set the tower and 4 drives up as a striped LVM Volume Group following this guide. This will give me ~75% useable space with the rest reserved for snapshots.

Make several Logical Volumes to organize files and control access by different users.

Format all as ext4.

Share the files on the network with NFS server/client setup.

Edited February 27, 2013 by amenditman

[securitybreach]

Sounds like a great plan!!