Desktop synchronization of files, with minimal side effects or bloat

[brainchild]

I very much would like to find a solution, for easily synchronizing with the local desktop a selection of folders in a remote share hosted on a DiskStation, that is more robust, resilient, and automated simply than file copies through file sharing or Rsync.

I have wanted to do so through a means that follows the share layout, user accounts, and file permissions of the Synology device.

I have understood the solution provided by the vendor is Synology Drive, but it is a rather bloated solution that comes with many unwanted effects. For example, it natively supports file versioning, which is unsuitable for many uses. Although versioning may be disabled, other effects may not be. For example, one effect I discovered by accident and that has created many problems until I did so, is of case sensitivity being changed by Drive for an entire share from a POSIX style to a Windows style.

So far, I have found no adequate solution to meet my needs. What other options might be available?

 

[EAZ1964]

For a desktop, the easiest one is not to share at all. Work like companies do:

Just move all local files to the NAS, work from a network share via share mapping.

If needed, create a backup task or share from the NAS to another device, cloud or other disk/desktop and understand that a sync without versioning is the worst type of data integrity apart from single storage.

 

[brainchild]

Just move all local files to the NAS, work from a network share via share mapping.

If needed, create a backup task or share from the NAS to another device, cloud or other disk/desktop and understand that a sync without versioning is the worst type of data integrity apart from single storage.

For use with wireless or portable devices, making local copies of data sets is essential.

For generally one-directional syncing (storage device to user device), versioning is not necessary, but it is invaluable to create and to destroy local copies of various subtrees from a remote share without affecting the remote data, and without manually administrating the various copies of the data set.

 

[EAZ1964]

wireless devices were not mentioned in your question so I left them out of my feedback.
You may have a good reason to ask for a sync, please explain why you prefer to sync.
A sync is not a backup, once the source is affected or accidentally deleted, the sync is hit milliseconds later.
If you just mean a copy from the Nas to the local device, you can setup a hyperbackup task to copy to the desktop using Rsync on the target.

 

[brainchild]

wireless devices were not mentioned in your question so I left them out of my feedback.
You may have a good reason to ask for a sync, please explain why you prefer to sync.

I want certain directories and their contents, including subdirectories, to be kept synchronized with files stored locally on a client running a desktop operating system, such as one based on Linux.

I would prefer a solution that supports two-way synchronization, but mostly the client will not be making modifications.

I did not mention wireless devices. I simply explained the kind of solution needed.

DiskStation seems to come with a variety of baggage and restrictions, but it is close to the kind of solution needed.

 

[Telos]

Have you considered Syncthing?

 

[brainchild]

Have you considered Syncthing?

Yes, and the use case is quite different.

 

[Telos]

I want certain directories and their contents, including subdirectories, to be kept synchronized with files stored locally on a client running a desktop operating system, such as one based on Linux.

I would prefer a solution that supports two-way synchronization, but mostly the client will not be making modifications.

That's what I'm doing with Syncthing.

the use case is quite different.

OK. I don't understand, but I'll step away now.

 

[Deleted member 5784]

Yes, and the use case is quite different.

Can you explain further? Syncthing appears to do everything you've asked for...

 

[brainchild]

Can you explain further? Syncthing appears to do everything you've asked for...

Syncthing does not follow the user, permissions, or credentials model of the host, and in fact, its operation is not particularly secure for multiple users on the same host.

A solution I would prefer would be one following the model of a cloud hosting service. The client would obtain a tree representation of hosted files, allowing the user to select specific directories to be copied from the server to the client, and maintained as synchronized, at least downstream.

 

[Deleted member 5784]

Syncthing does not follow the user, permissions, or credentials model of the host, and in fact, its operation is not particularly secure for multiple users on the same host.

Syncthing supports syncing of file permissions, file and directory owners & groups, and syncing of POSIX and NFS ACLs via exattribs.
(from here)

Device to device connections are encrypted using TLS 1.2 or 1.3. Does your envisaged usage include having untrusted clients syncing to your NAS?

I have no skin in the Syncthing game; it just seems to be an obvious candidate for your needs, even after the further elucidation of your latest post.

 

[EAZ1964]

I doubt if your expectations can (technically) be met.

While copying a file with all attributes and user/group permissions from the nas to eg windows, apple or linux host, how can a system guarantee that user permissions be retained? Possibly the NAS users and groups do not even exist on the new host.
File system attributes (BTRFS vs ext4 vs NTFS) may be different so the copy to a different file system causes loss of attributes.

Anyway, succes but I will step out as well, as the objective is not clear.
Any answer will be as good as the question

 

[brainchild]

Syncthing supports syncing of file permissions…

Device to device connections are encrypted using TLS…

While copying a file with all attributes and user/group permissions from the nas to eg windows, apple or linux host, how can a system guarantee that user permissions be retained?

The issues are not copying permissions from one device to another, or copying over secure sockets.

The issues are respecting login credentials and respecting access permissions provided by the storage system.

Access to files should be based on providing credentials for a user account on the storage system, and on enforcement of user permissions against any files being accessed.

 

[fredbert]

I’ve tried to understand what’s the problem that’s trying to be solved but really you’ve explained it like a government technical requirements specification. And I’ve seen those and it’s anyone’s guess as to the right solution.

Maybe you can outline the use cases you are trying to achieve. Such like how users and devices are to behave and how the data is copied and accessed.

My impression is that your need for access permission preservation is so that multiple local PC/Mac/Linux users can access to local synced data, but only what the NAS would’ve permitted. This implies some link between local device and NAS accounts, plus data synchronisation has to be performed by a local privileged account using login to NAS with a DSM admin account… otherwise how is access controlled on the local device? I’m not sure how all this would or could happen.

 

[brainchild]

My impression is that your need for access permission preservation is so that multiple local PC/Mac/Linux users can access to local synced data

As explained, the need is not to preserve permissions for files copied locally, only that access to files on the remote store by the client is based on actual login credentials and file permissions on the remote store.

Think of cloud solutions such as Nextcloud or Google Drive for a close analogy.

The client authenticates with the server using login credentials, and then may access only files that are owned by the authenticated user or shared with the user by another user on the system.

Synchthing does not query the authentication system of the host or match user authentication against login credentials or file permissions. In fact, I believe Syncthing is not even using a client-server model.

 

[brainchild]

Sorry, but I think the reasons are clear why the suggestion is not a constructive one. I appreciate your participation, but I prefer that the contributions be kept relevant.

 

[Deleted member 5784]

Think of cloud solutions such as Nextcloud or Google Drive for a close analogy.

The client authenticates with the server using login credentials, and then may access only files that are owned by the authenticated user or shared with the user by another user on the system.

It is possible to install eg Nextcloud on a synology NAS using Docker to install the Nextcloud, Apache, MariaDB etc containers. Google points to several guides eg here, here, und auf Deutsch.

However, as you've no doubt realised the point of a Syno vs, say, an open linux server is that one is kind of encouraged to use Synology's own proprietary services and solutions for this kind of thing, in this case Drive. It's not that you can't concoct other more open solutions, its just that implementing them is frequently more complicated and tortuous than doing so on an open linux platform. Hence docker for everything.

Personally, if using a Synology as the sync server and Drive wasn't suitable, for simplicty's sake i'd probably stick to using rsync on the client(s) and scripting whatever schedule etc you need. I'm not sure you've specified in your posts above which client OSs need to be accomodated by any solution? If phones are involved that could complicate things...

 

[fredbert]

The client authenticates with the server using login credentials, and then may access only files that are owned by the authenticated user or shared with the user by another user on the system.

That would be ’login credentials’ on the NAS that have no relationship to the local device login session? Meaning, for example, I log onto my Mac and I use one of the DSM accounts I know to create a SMB mount in Finder. That the local and DSM accounts have no technical relationship, only purely social.

What ’m being rather dim about is understanding how you see the local client (SMB, rsync, SFTP, Drive, whatever) being somehow getting access to data that the server on the NAS won’t allow: the NAS server’s apply access controls based on whatever criteria they employ, which may include some part of that inherited from file system permissions.

Is the thing you are looking for is on-demand sync? So only files are resistant on the client device when they are requested by the local user. This seems to be now what you’re looking for, thanks for the clarification on the single-user requirement.

Really it’s Synology Drive if you’re wanting the easiest solution to deploy. If you’ve found there are problems with how Drive is working then I’d suggest you open a ticket with Synology Support as you may have discovered unintended behaviour that needs to be fixed.

An earlier solution that was mentioned would be to utilise OneDrive/Google Drive to be the cloud to device mechanism. Then use Cloud Sync to retain the oneDrive/Google Drive content up to date on the NAS. Of course that assumes you are happy to use these public cloud services. For me I still have a small Dropbox account that I Cloud Sync, then use Drive to provide access to it on my Macs and mobile devices. I find the Drive client to be a lot less bloated than the RAM hungry Dropbox agent.

The older deployment of Drive used /home/Drive as the user’s home Drive folder. At some point new installs started to use /home. I still have /home/Drive since it means less work and content to manage by Drive.

 

[brainchild]

That would be ’login credentials’ on the NAS that have no relationship to the local device login session?

The local device and the storage device have separate user accounts and enforce different login credentials, of course, unless they are configured to participate in some shared domain.

Sorry, but I am not understanding why the particular question would be a point of confusion.

 

[brainchild]

It is possible to install eg Nextcloud on a synology NAS using Docker...

...the point of a Syno vs, say, an open linux server is that one is kind of encouraged to use Synology's own proprietary services and solutions for this kind of thing, in this case Drive...

Nextcloud uses its own storage backend, account credentials, and file permissions. It would be deeply unsuitable.

I agree that Drive may appear as the correct solution, but it seems to have some extremely serious flaws and constraints.

-- post merged: 28. Aug 2023 --

Really it’s Synology Drive if you’re wanting the easiest solution to deploy. If you’ve found there are problems with how Drive is working then I’d suggest you open a ticket with Synology Support as you may have discovered unintended behaviour that needs to be fixed.

Some time ago I had a ticket open for almost a week, until we realized that an Rsync job was behaving erratically due to Drive changing the case sensitivity of the share.

After, I disabled Drive, and never returned to it.

The technician insisted the behavior is a feature. Intended or not, it is not suitable.