One month with TrueNAS Scale (now in RC-1 stage) and Kubernetes

[WST16]

I have to admit that I've looked into TrueNas long before you started posting, but I dismissed it (for personal use). If I recall correctly, my reasons were that it's more suited for an IT department, a company, rather than for personal use (might be the wrong conclusion).

 

[Coop777]

I will say if push comes to shove and the locking down rolls down to the NAS models I run that I would probably migrate over to TrueNAS. For now, the saving grace of Synology is a provided supported product that doesn't require alot to run it and thus far I haven't had any real issues other than the C2000 processor failure, but they can not be blamed for that.

 

[jeyare]

the question of whether you looked at:
- TrueNAS Core (FreeNAS) based on FreeBSD and running from 2009 (first release)
or
- TrueNAS Scale, which is in DEV since 1/2021 based on Debian

These are two fundamentally different products with two common denominators: OpenZFS and Angular-based UI.
FreeNAS/Core is based on community support, while Scale is based on Community & Professional Support (up to 24x7x365).

here is a detailed comparison:

Help Me Choose Between TrueNAS CORE, SCALE, or Enterprise

TrueNAS has three editions to choose from. Let us help you choose which edition of TrueNAS is right for you.

www.truenas.com

which isn't valid in one row (Scale part):
Linux Containers (Docker), Pods (K8's) support

neither Docker containers, nor k8 (just k3)

 

[Coop777]

A friend of mine is running FreeNAS, I believe and likes it. I have not done the deep dive into all of TrueNAS - something to keep an eye on.

 

[jeyare]

I would be careful with this wording:

Synology is a provided supported product

in recent 2 years, this support has deteriorated not only in terms of support for 3rd part drives/RAM, but also the 1st level support itself. My optics may be distorted, but when I write to Syno support, it's not a trivial matter. And the solution score is well below 50%.
DSM is so overwhelmed by old things that running it on a new Linux core is a dream rather than a reality. This is related to the support of new CPUs, MoBo architectures/components, PCIe buses, ...
One thing I have to admit is that it used to be a joy to do a new DSM upgrade (still on 6v on my all NASes). Today, after the upgrade, the UPS daemon does not work, especially on the DSM7 UPS, disks, ...

 

[one-eyed-king]

Without Helm support.

What is Helm support? Since Helm3 you don't need to initialize a backend component - access to the kubeapi and a sufficient set of permissiosn should be enough to use Helm3.

On a second thought I guess you are talking about the limitation that the TrueCharts APP doesn't support custom deployments based on Helm3? Would it help to run your own app catalog using something like Kubeapps, deploy your applications in Kubernetes

 

[Coop777]

I would be careful with this wording:

in recent 2 years, this support has deteriorated not only in terms of support for 3rd part drives/RAM, but also the 1st level support itself. My optics may be distorted, but when I write to Syno support, it's not a trivial matter. And the solution score is well below 50%.
DSM is so overwhelmed by old things that running it on a new Linux core is a dream rather than a reality. This is related to the support of new CPUs, MoBo architectures/components, PCIe buses, ...
One thing I have to admit is that it used to be a joy to do a new DSM upgrade (still on 6v on my all NASes). Today, after the upgrade, the UPS daemon does not work, especially on the DSM7 UPS, disks, ...

fair enough. I admit I haven't had to contact syno support much since the 1815+ c2000 fail, and haven't been bit by DSM 7 upgrades.

 

[jeyare]

What is Helm support? Since Helm3 you don't need to initialize a backend component - access to the kubeapi and a sufficient set of permissiosn should be enough to use Helm3.

On a second thought I guess you are talking about the limitation that the TrueCharts APP doesn't support custom deployments based on Helm3? Would it help to run your own app catalog using something like Kubeapps, deploy your applications in Kubernetes

yes, yes … till the next update of the Scale system

 

[one-eyed-king]

uh,, so they basicly kill the cluster and deploy everything from scratch and whatever is not in a pv is lost?

 

[jeyare]

@one-eyed-king
yeap,
they just support GUI deployment trough the TrueCharts APP catalogue

no way to use Helm in other ways as by installing, so:
You need first deploy Helm, then Helmcharts
You can use Helmcharts
You can create any Pod and all the customization
everything deployed by this way is lost after next upgrade

all is summarized in this sentence from the TrueNAS forum:

Using Helm commands is 100% possible, the command is just not enabled by default because iX doesn't want to support users using it.

-- post merged: 12. May 2022 --

there is still unanswered question:
- who is the target of the Scale product?

If average homelab admin, then everything is fine. You can install your container trough TrueChart APP with weak customization, strictly with the defined template contains a default values (some you can’t change). Source of the container is unclear and you will be during the lifespan of the container dependent from two sources:
- creator of the container
and
- TrueCharts - group of randomly composed people.

It means that you can’t decide when you will use what version of the container publicly available - you need:
- wait for the TrueCharts people do deploy the container to their template
- wait for the TrueCharts people for new version of the container deployment

and here is the most important point:
you can’t change setup of your Pod by CLI. Just GUI way is enabled.

Is it really in line with an Enterprise expectation?

 

[one-eyed-king]

I wouldn't concider the ootb missing helm cli command on the node not as missing helm support.

But it sounds like the original problem is that the cluster state (which normaly is persistant in a etcd3 cluster) is not persistent, which makes helm deployments ephemeral. Having the configmaps/secrets that helm uses to store helm state indeed makes it unbearable to use.

Idiots The usual game: take something that is proven to be excellt. Bend it to force your opinion/philosophy on it as long as the deviation from the standard makes it a pain to use. scarry....

-- post merged: 12. May 2022 --

Is it really in line with an Enterprise expectation?

Definitly not.

Not storing cluster state actualy says a lot: it is not ment to be used by the user, it just happend to be the runtime for their truechart apps.

 

[jeyare]

Even worse,
you are limited by the default template setup (encapsulation of the container) with values (another yaml). Just for example - you can’t run Unifi controller from Linuxservers distro, only from Jacobalberty (which isn’t bad). But you can’t define all the possible Enviro variables, because TrueCharts values.yaml contains only few of them (amateurs or idiots). And in the TrueCharts GUI you can see only what is stored in the values.yaml.

Yes, it is great attitude for folks need ASAP run “SQLite browser” in k3s. But out of my expectations.

Here is a list of supported TrueCharts:

charts/charts/stable at master · truecharts/charts

Community Helm Chart Repository. Contribute to truecharts/charts development by creating an account on GitHub.

github.com

you can’t find there more than 10% useful for SMB, unless it is the main purpose of the SMB to operate torrent download or similar pearls.
Great.

 

[jeyare]

I pointed out to iXsystems that their partner TrueCharts also has in its catalogue of charts some items that should not exist there from the point of view of basic safety principles - they contain Critical CVEs. I suggested to them that the CVE check status needs to be pinned in the catalogue so that the user knows in advance what he is installing - or simply they don't allow such charts to appear in the list by design. That was in November 2021.

Today is May 2022.
The TrueCharts catalogue contains a CVE check. Great.
The TrueCharts catalogue still contains containers that contain Critical CVEs. Bad.
Example - Airsonic chart:

Security Overview - TrueCharts

Project Documentation for TrueCharts

truecharts.org

Reason:
Following the TrueCharts github - Chart template (chart.yaml) we can read the sources of the container:

apps/Chart.yaml at master · truecharts/apps

Community App Catalog for TrueNAS SCALE. Contribute to truecharts/apps development by creating an account on GitHub.

github.com

it is Airsonic itself
or linuxserver

However, the Airsonic container source (image) for the TrueCharts was lastly updated by the developer 2 years ago:

GitHub - airsonic/airsonic: Airsonic, a Free and Open Source community driven media server (fork of Subsonic and Libresonic)

:satellite: :cloud: :notes:Airsonic, a Free and Open Source community driven media server (fork of Subsonic and Libresonic) - GitHub - airsonic/airsonic: Airsonic, a Free and Open Source community...

github.com

with a big message from the developers:

Airsonic isn't maintained anymore, you should migrate to airsonic-advanced instead.​

so how gents from the TrueCharts can change this critical CVE? No way.
In November 2021, I advised them to remove such containers from the list. Nothing like that happened. Because the list must be large, otherwise they do not attract users. this will cause low interest in the Scale (from the usage of containers). After all, 80% of users do not understand this and who would still take care of security issues? Great approach, though?

But here the situation is changing because:
- it is no longer just a random search for containers, which the user installs himself
- this is already an umbrella activity to provide deployment through a trusted partner (TrueCharts) of the Scale system (iXsystems)
That's a dramatic difference.

Btw:
this container is also available from linuxservers. But with the same message:

This image is deprecated. We will not offer support for this image and it will not be updated. We recommend our airsonic-advanced image instead: GitHub - linuxserver/docker-airsonic-advanced

Docker Hub

But the TrueCharts isn't pointed to the new container's Airsonic-Advanced image.

Conclusion:
If this is the only iXsystems partner for the massive deployment of Kubernetes through the new Scale product, then it's almost a road to hell. So much for responsibility on both sides (iXsystems + TrueCharts).
Or an excellent example of flexibility from TrueCharts.
In a world where the user is not limited, what source to use for container deployment would be very easy to solve.
However, In a world where you need to use TrueCharts only, it is a disaster.

 

[aconnor]

I actually registered an account to reply to this thread.
Agree that TrueNAS scale has a lot of potential, but some interesting decisions have been made.

For me, I have 2x Xeon powered Synology units- I'd really like to spin up a VM on each one, then maybe a more powerful VM on my own hardware running TrueNAS Scale and have a k3s cluster.

But TrueNAS Scale has only a single host version of k3s and it's unclear if you can add to it...
And the Truecharts support person was actively hostile to me when I asked questions- like '30 years of support and this stands out' type of hostile

 

[jeyare]

@aconnor
TrueCharts path to containerization is incompatible with my view of flexibility / security / freedom
When I openly and freely wrote to them where they have security holes, I became a dangerous creature. It's light years away from the approach of the LinuxServer.io guys.