Saturday, December 26, 2020

A painless self-hosted Git service

Remember how a part of my NAS setup was to host my own Git server?

Well that forced me to review options and I stumble into Gitea.  A extremely sexy GitHub replica with more spices and lighter profile.



It was a breeze to set it up and it has all the goodies we have grown to love in GitHub.  Host it on your own PC, NAS, docker container, whatever, it is just great.  We have uploaded our archive repository on it (around 600MB of just code) and it's handling with no issues.

You can control organizations, teams, users, private/public profiles, setup the configuration in PostgreSQL, MySQL or MSSQL it is all possible.

So, if you want to avoid dependencies on external companies or avoid future charges for it.  Browse around on the Gitea site.



Wednesday, December 23, 2020

A Walk in the Cloud


There will be a moment in your life when you have enough gadgets and computers at home that you will start to worry about what will happen if you lose one of them.  Everybody does the IP68 test of throwing the phone to the toilet, so suddenly those photos, contacts and things that you treasure may go to Alice in Borderland and you definitely don't want that.

My phone wasn't IP68 complaint and my scripted backups and online cloud settings were not enough to backup all my data (I hate paying yearly subscriptions.. ok fine, movies are an exception) so I decided to do a Keanu Reaves and do a Walk in the Clouds by setting up my own.

The objective

1. Phones.  All contacts, photos and important files to be backed up.

2. PCs. Backup all virtual machines, databases on a set schedule.

3. Photos.  Terabytes of pictures stored everywhere to be uploaded and kept safely on the NAS.

4. Clouds.  Get out of it. Google Drive, Microsoft One Drive, DropBox.  Back up all their data into the NAS and keep it synch so any new stuff added there to be automatically copied back home.

5. Email.  Extract all the emails from my gmail, outlook and other third party mail providers and kept it in a searchable format in case of a problem with any of my accounts.

6. Git. A git server where all my code will reside.

7. Space. I need around 8 TBs of space, probably I will just end up using 3/4 but I'm certain that once space is not an issue, things will star to grow, plus the more gadgets we add the more data we collect.

8. Lots of redundancy. Minimum one, better two disk failure redundancy.


The Chosen One (Is that you Anakin?)

Synology DS1520+


Why this one? well you have plenty of options out there, QNAP, ASUSTOR or built your own NAS but life is short and I wanted the most user friendly software interface to interact with your NAS system out there. I wanted to be up and running in no time with all my data safe and available on the shortest time possible.

That being said, I needed 2 disk redundancy, we are talking about a RAID 6 (or equivalent), so a 4 bay was a minimum. 5 bay is just a little future proofing on my part.

The Setup

1. 4 X 4TB Ironwolf NAS disks.

This may imply a juicy 16TB of space but I ended using Synology's homebrew RAID 6 variant called SHR 2.  Which combines RAID 6 with the flexibility of adding extra HDDs of mixed capacity and use the additional space once I have all the HDDs of the same size without having to rebuild the array.  This means giving up 2 full disks for the sake of redundancy.  Long story short, final space available 7.6TB. So that extra slot is key, because later on I can add an extra 4TBs without penalization.

2. Dual 1Gb lines to my router.  The Synology supports link aggregation/bonding so I can have up to 2Gb output in absense of a higher bandwidth wiring around the house and a router that supports it.

3. 1TB WD NVME module for read-only cache purposes.  Useful for small constant files aka photos, which is the bulk of the load of this NAS.

4. 8 GB RAM, max out which could be an issue but we'll see.

5. Intel Celeron J4125.  Yes, I know.  There are Ryzen servers but others specs go up and as such price do to.  So far my use cases are not on the range of having to go up in CPU power being mainly a storage solution.

The Result 



This is where the Synology ecosystem shines. 

1. Phones.  I was able to setup Synology Moments to backup all our phone photo folders and download almost immediately any new photo that I take.  Moments itself is worth a different post due to some extremely good features that I appreciate on handling photos.

2. Contacts. It was easy to setup a contacts server which synchronizes with any popular android vCard software.  It also allowed me to access my Google Drive and Outlook accounts and synchronize with them centralizing all my contacts.  It was really a matter of a couple of clicks.

3. Photo Archives.  Drag and drop thousands of photos into the folder structure of the NAS and after being copied, Moments will indexed them and apply some AI magic to have them ready to be seen and share on your phone or any other device.

4. Email.  You can setup a web mail client on your personal cloud and enable pop3/Imap features to access your other email accounts downloading all their content into your inbox without deleting the original email.

5. Git server.  Still having trouble with it.  SSH is mandatory and I still need to play a bit more with the popular GUIs like TortoiseGIT, SourceTree, etc.  Console is there, but its 2020, why do I want to use the console If I can avoid it.

6. VM backups, DB backups.  Integration with MSSQL a piece of cake.  VM Backups, still working on it.  I don't use vmware vSphere or Hyper-v  (I'm more of a xcp-ng guy (xenserver)) so going slow there.

Conclusion

How on Earth I was surviving without a NAS at home.  No idea.  I'll keep some of my data on Google, MS just for ease of access but otherwise, I'm walking in My cloud now.


WARNING

A NAS is not a backup.  You still need to have an additional layer to safe guard your data. 


A painless self-hosted Git service

Remember how a part of my NAS setup was to host my own Git server? Well that forced me to review options and I stumble into Gitea .  A extr...