I have done the installation, network configuration (securing ssh and Cockpit), so I'll write the first section on networking with the notes I've taken during the test. Then I'll move on to testing storage configuration.

To sum up the next steps,

• Fedora server installation: testing and config done. Draft by 31 Dec
• Network configuration: testing and config done. Draft by 31 Dec
• Assemble Pi cluster case: Done.
• SSD configuration: Testing by 31 Dec, draft by 31 Jan
• S3 cloud storage configuration for offsite backup: Testing by 31 Jan
• End-to-end testing and data transfer: TBD depending on above.
• Reliability test: TBD depending on above.
• Final draft: TBD depending on above.

@hankuoffroad Thanks for the update!

On S3 cloud storage, I managed to shortlist third-party cloud backup services and learned the pros and cons of each option.

• Public cloud / Open Stack: Cost efficient and a range of backup services/storage types/lifecycle. It seems like a feasible option to start with. Well documented and community support. Free credit offered. Migration supported - multi-cloud strategy.
• Self-hosted private cloud / Open Stack: Overwhelmed by configuration. Can't get my head around with this business model, yet.

As this is a home solution, I'd rather have two offsite backups, which do not require RAID maintenance. To sum up the design concept,
A local drive connected to server (separate from source data in the workstation)
backup 1: rdiff-backup with a public cloud service (accessible in x minutes/hours)
backup 2: rdiff-backup with a public cloud service (long term storage - cheaper than the above)
backup 3: onsite backup (USB drive using rdiff-backup and disk encryption). Due to travel restrictions, this is optional. Struggling where to put it away. Any thoughts?

I dropped RAID in the design. No redundancy is required. I want low maintenance - taking care of two separate drives (different types of disks) plus monitoring cloud backup.

Sounds good to me. I'm not sure I understand your question about where to put the onsite backup material. I guess if it is optional or only tangentially related to the rest of the article it should go at the end of the article.

Thanks!

A sketchy summary/table of the content of the article is here. I see there are the Fedora articles that cover most of my setup, so I cited the links. I will focus on the testing of the backup and restoration process after the festive break. Thanks!

3-2-1 Backup plan with Fedora ARM server

• Data backup and restoration for personal users

Fedora Server Edition works on Single Board Computers (SBC) like Raspberry Pi. This article is aimed at data backup and restoration for personal users who want to take advantage of a solid server system and built-in tools like Cockpit.

The objective of a data backup plan for personal users is;
- Prevention of data loss with backup and restoration plan.
- Low setup and running cost with the small form factor SBC.
- Use Cockpit for maintenance and monitoring.

The design concept of data backup flow is;
Keep copies on the local drive, in the cloud (file and system backup), and also offline on a USB drive from time to time.

• Pre-requisites;
  Read, understood, and practiced requirements for server installation and administration.
  Fedora Docs - part 1
  Fedora Docs - part 2

An SBC (Single Board Computer), tested for Fedora Linux. Check hardware status here.
Fedora Docs
Fedora ARM server raw image & ARM image installer
A choice of cloud backup services, which adhere to open source standards
1 SSD drive that is powered by a USB port
Your computer for remote access to the server and for local network file share
Optional: 1 external USB drive

With this setup, I chose Raspberry Pi 3B+ because of price, availability, and low power draw. The number of years on the market means there are more users, use cases, and support articles available.

• Permissions and firewalld
• Switch off all services not related to storage server using Cockpit
• Storage configuration: Create partition table (Cockpit)
• Storage configuration: NFS
• Backup: rdiff-backup
• Backup: Guard against accidental deletions or misconfigurations
• Backup: Backup to cloud
• Restoring from backups (Unfinished)
• System backup (Not started)
• Reporting with Cockpit (Unfinished)

The markdown notation I used wasn't correctly rendered. Will check it on the main article. Thanks.

For the originality of my article and test, I need a duplicate Pi for failover test, so I can quickly swap it with the recovery image. I found out Pi 4 is sold out in the UK where I live, so there will be a bit of delay in the testing of the recovery process.

I aim to upload a polished draft to Pagure by end of February. I enjoy every bit of experiment and working with .dot files. I'm still amazed by the fact the server OS runs on a Pi 24/7 and responds to me on ssh.

My colleague joked about the pic I shared - it looks like you are building bitcoin server mining!

@hankuoffroad :-) Thank you for the status update.

Remember, rather then uploading a draft to Pagure, open a new article in WordPress. Select "Posts>Add New" from the menu at the left of the page to get started.

Remember, rather then uploading a draft to Pagure, open a new article in WordPress. Select "Posts>Add New" from the menu at the left of the page to get started.

Sure, I did a rough draft in WordPress just now to have a mental snapshot and will add more (& write in a full sentence) when each stage of the test is done. I set it to holding pen status for now.

@hankuoffroad there may be a chance that WordPress won't let you make further revisions to an article that is in the "holding pending review" status. If you find that that is the case, just let us know with a comment here on the card and we can move your article back to "draft" status.

@glb It works fine when I made revisions in 'holding pen', which I guessed the status means when the writer starts to write and the content is in the early stage.

You can certainly use it that way if you want. I just wasn't sure how the WordPress permissions were set and I didn't want you to think that we intentionally locked you out of the article if that happened. I think those are just some default statuses that come with WordPress. We (the editors) don't really use them. We use Pagure instead.

Documenting the process revealed I was going back and forth between terminal, Cockpit, and back to the terminal for editing config, which appears messy. I paused further testing in January. My idea is to use Cockpit for the NFS file-sharing process.

I noticed there is development and testing planned for Cockpit file sharing.
https://fedoraproject.org/wiki/Changes/cockpit-file-sharing-2-4-1-5

Can I participate in testing so I can make the NFS share more fluid and document the process before the F36 release?

@hankuoffroad thanks for the update.

I noticed there is development and testing planned for Cockpit file sharing.
https://fedoraproject.org/wiki/Changes/cockpit-file-sharing-2-4-1-5

Can I participate in testing so I can make the NFS share more fluid and document the process before the F36 release?

It looks like @sgallagh and @pboy own that change request. They would be the ones to contact for information on how you can test out the new Cockpit plugin. They've both written for Fedora Magazine before. So they should be familiar with the article workflow and with participating in the comments on these cards. I'm sure they'd be happy to hear that you plan to advertise the new feature they've been working on too.

Testing cockpit file sharing module is already underway by Server WG @pboy, which is now part of F36. So I would track the release schedule and get my hands-on as soon as F36 beta testing is available.

To put this publication schedule back on track, I'll go ahead with the NFS setup with the configuration file, not Cockpit. I'll use Cockpit for server monitoring, so the backup workflow is not dependent on Cockpit.

After four months of running Rpi nonstop, I'm now okay to soldier on the rest of the recovery test such as accidental deletion of source files and recovery from another laptop. I have got a replacement Rpi for hot-swapping amid the global chip shortage. Recently, my Pi server survived powercut, which is one of the key disaster scenarios.

I'll let you know when a first draft is ready. Thanks.

Thank you for the update

The latest kernel update (5.16.x) appears to break the NFS mount as reported in the bug tracker.
I'll track the next update if that comes with fixes. Please bear with me and I'll let you know if I complete the test bug-free or find a workaround.

@hankuoffroad Thank you for the update.

I just managed to do the NFS mount for my R Pi 3B+ and Silverblue workstation using Cockpit. I ssh into shared folders in the NFS server via the GNOME Files. Cockpit really makes my life easier. For me, NFS is much simpler to set up than Samba.

I aim to complete the backup/recovery test by 3 April. I will create an updated diagram after that.
I'm pretty sure a draft will be ready by 10 April for your review.

It took longer than I thought, sorry.

@hankuoffroad Glad you are forging ahead. Thanks for the update.

The first draft is updated. Some sections need writing in full sentences.
By 10 April, I'll add the restoration process and monitoring with Cockpit.

A minor change:
NFS: I omitted NFS here as it is the article on its own. Raspberry Pi works as a file/backup server rather than an NFS mount/file sharing. I thought it would be better to focus on core features - ARM server, Cockpit.

Backup utility: I opted for rsync rather than rdiff-backup.

Supported hardware list: Link pointing to the Docs, not wiki

Question
<username>@<username> in inline codes and -- are not rendered correctly.
-a, --archive
-z, --compress
-P, --progress

Can you advise how to fix it?
I changed inline code format to normal paragraph.

@rlengland Draft submitted for your review.

Thumbnail image candidates: How about using Fedora server (orange color) and Raspberry Pi logos on a plain background
Links for SVG images here

Fedora server: server-full_light-bg.svg
https://fedoraproject.org/wiki/File:Edition-server-full_light-bg.png

Raspberry Pi SVG/transparent PNG
https://freebiesupply.com/logos/raspberry-pi-logo/
https://www.svgrepo.com/svg/303239/raspberry-pi-logo

@rlengland are you working on this one? It looks like Hanku had it ready for review about a week ago.

@glb I have not been working on this. But I will.
Unfortunately, a good deal of this will be a dry-lab exercise since I don't have the hardware to test on.

@hankuoffroad I've done an edit pass on your article and it is close to being ready to publish, but I have some suggestions for some "tweaks" that might improve it.

• I feel that the section "ssh key-gen in the file server and transfer to cloud storage" could use some further explanation about what is happening here. Where the "keys" come from and where they are going

• In the section titled "Restore a directory from cloud storage" can you check the entry? It appears to be a duplicate from the previous section.

• Under "Archive Personal Data" you indicate that you should "Go to "Sharing" and toggle the slider to enable sharing in the graphical file manager." What is the graphical file manager you are referring to? Can you provide some further detail on this? I'm not seeing this in the file manager for GNOME.

I did some minor language editing but it would be good if you can read it over to make certain I haven't changed anything critical. We can schedule soon if you can respond to the points above.

I'll put together a feature image for the article. Thanks for all your work.

Superb! I've revised the sections that require clarity and further explanation. Checked x 10 times :) for mistakes, some of which were caused by tunnel-visioned after months of trial and error.

Thanks for a feature image.

I'm indebted to @rlengland @pboy @glb @mattdm for help/advice. Really enjoyed the experience of working with the server installation for the first time, navigating contents in the Fedora Project, and some from Red Hat. It's all there.

I can't forget the generosity and patience from Ask Fedora, discussions, and the Magazine writers and editor for my rudimentary questions.

I had moved this to next Friday (the 22nd) because I thought you needed more time to work on it. But if it is ready to go now, you can still schedule it for this Friday (the 15th) if you want to. There is nothing else currently scheduled for that slot.

FYI,
gb

I'm just glancing over the article now and I see something that doesn't look quite right to me.

firewall-cmd --add-rich-rule='rule family=ipv4 source address=/24 service name=ssh log prefix="SSH Logs" level="notice" accept'

Should there be an address or a placeholder in front of the "/24"?

I'm just glancing over the article now and I see something that doesn't look quite right to me.

firewall-cmd --add-rich-rule='rule family=ipv4 source address=/24 service name=ssh log prefix="SSH Logs" level="notice" accept'

Should there be an address or a placeholder in front of the "/24"?

I missed that when the formatting removed < placeholder >
Thank you.

I missed that when the formatting removed < placeholder >

Yeah I know. WordPress can be difficult to work with. For future reference, I've found that that sort of auto-formatting can be avoided by clicking on the vertical ellipsis (⋮) for the block and selecting "Edit as HTML" before pasting the pre-formatted content (just be careful not to overwrite the <pre> tags). After you've pasted the content, you can switch back to the "Edit visually" mode.

FYI,
gb

I had moved this to next Friday (the 22nd) because I thought you needed more time to work on it. But if it is ready to go now, you can still schedule it for this Friday (the 15th) if you want to. There is nothing else currently scheduled for that slot.

FYI,
gb

Finally, I spotted rsync -azP where the option to denote the progress of rsync should be P rather than p (preserve permissions).

Other than that, the article is ready to be published now. Can you schedule it for the 15th, if possible? Thanks.

Scheduled for 15 April

I see the potential for my first Pi server project to be extended to the Redis cache server for storing historical data (logs) and displaying analytics. I'll keep it for the next project.

The Pi server is running since the first boot on 15 Nov 2021.
10 S/W updates have been automatically done.

$ last reboot
reboot system boot 5.16.19-200.fc35 Wed Apr 6 00:59 still running
(....)
reboot system boot 5.15.12-200.fc35 Mon Nov 15 00:00 - 23:57 (53+23:56)