Mac clients - SMB cache issues

[turing]

@turing I would not recommend the use of AFP - it is depreciated and not maintained.

Suggest entering this command via macOS Terminal:

defaults write com.apple.desktopservices DSDontWriteNetworkStores -bool TRUE

Then log out and back in again. This may fix your SMB issues so you can get off AFP.

️

Hi Robbie:
I will try that.
I went SMB initially for exactly the reason that you mentioned... but the severe flakiness drove me back to AFP.

 

[turing]

@turing I would not recommend the use of AFP - it is depreciated and not maintained.

Suggest entering this command via macOS Terminal:

defaults write com.apple.desktopservices DSDontWriteNetworkStores -bool TRUE

Then log out and back in again. This may fix your SMB issues so you can get off AFP.

️

I followed your advice and using SMB is, basically, not going to work for me - if a folder has more than a couple of dozen files in it, it takes so long to deliver the list of files that the applications that I use time out and consider the connection dead.
That is, it can take several minutes for the NAS to respond to the request for the file list and, since the applications that I use are connected indirectly to the NAS through a network port, they bail.
Even though AFP is deprecated and not maintained, it is quite capable of producing a file list in a matter of seconds and thus works for me.

 

[Robbie]

Ok but make sure you have a recovery plan if your data goes sideways.

I still find it difficult to think the SMB issue is incurable; I have M1, M1Pro and M1Max machines (plus Intel) all running perfectly on SMB, from 10GbE card to aggregation to vanilla 1GbE. No delays, slowdowns or even hiccups from any of my 3 Syno NASes nor any difficult running SMB between macOS and Windows.

When I thought I had an SMB issue it invariably turned out to be either a hardware issue or a setting corruption.

️

 

[wblew]

I was having what seems like similar problems:
- finder was very slow to open folders on mounted SMB shares
- mounted SMB shares would sometimes disappear randomly
- after selecting a shared folder within the MacOS's "Connect to Server..." dialog, the folder, then its contents, would each take a long time to appear.

I have made two changes that have dramatically improved things. With these changes, my folders open immediately, and I am getting sustained line speed file transfers to/from my Synology NAS and my iMac 2020 running the latest Mojave update.

While your "mileage may very", for what its worth, here they are:

First issue: my Mac could not reliably determine the address of the Synology server, given its server DNS name, whether it was based on Bonjour (ended in .local), or was DNS based, (was an AD domain, e.g. nas.example.com). Thus changes A) and B) below... (This was despite SMB1, aka NetBIOS over TCP, being disabled on my MacOS Mojave installation, go figure...).

A) On the Synology, check its "Enable Local Master Browser" setting:

I sort directories on the server, as my DS1621+ is a beast, given my current network is limited to 1 Gbps links. Also, the "Local" in Master Browser, means: local to the LAN's subnet. Given the NETBIOS browsing was based on broadcasts, multiple subnets gets nasty fast... and is an advanced topic.

The above comes with are two major caveats:

A1) The NAS MUST NOT be an Active Directory Domain Controller. Why? Samba cannot be both an AD DC, and Master Browser / authoritative WINS server, at the same time.

A2) When you have multiple samba servers, only ONE of the should be a Local Master Browser. Otherwise, they might hold constant elections in an effort to defeat each other, to become The one and only Local Master Browser.

B) Determine that NAS's IPv4 address, and make it the WINS server for your SMB client workstations. You can use a DHCPv4 option, or manually configure them on your Windows and MacOS boxes. There is also a setting for linux samba installations, etc.

Here is the Synology DSM setting, as an example:

Second major issue, despite my MacOS's network service order, having my wired ethernet connection higher than my WiFi network... the MacOS would frequently choose to route its SMB traffic over my WiFi network, rather than my clean ethernet wires. Hence I updated my MacOS SMB client settings.

C) Update your MacOS client's SMB settings as follows. The first two entries were defaults on my Mojave install. But the third option was added to never use the WiFi when there was a choice.

Edit your MacOS /etc/nsmb.conf file as follows:

# /etc/nsmb.conf
[default]

# Limit negotiation to SMB2/3 only.       NOTE: SMBv1 has broken security, and is a slow pig. Just disable it!
# 7 == 0111  SMB 1/2/3 should be enabled
# 6 == 0110  SMB 2/3 should be enabled
# 4 == 0100  SMB 3 should be enabled
protocol_vers_map=6

# No SMB1, so we disable NetBIOS.    NO, really disable SMBv1, along with NetBIOS over TCP, its transport.
port445=no_netbios

# The above are shipped as MacOS defaults.
# https://gist.github.com/jbfriedrich/49b186473486ac72c4fe194af01288be

# Prefer lower latency wired ethernet, over higher
# latency WiFi ethernet. Despite WiFi advertising
# higher speeds.
mc_prefer_wired=yes

Note: the MacOS's man nsmb.conf Terminal command is your friend. It's well written, and seems complete.

As a point of interest... all network file sharing protocols are very sensitive to both latency and reliable packet delivery. It is of particular importance during session setup (mounting the share), directory listings (whether ls command, or Finder window), or session teardown (i.e. un-mounting).

One last point.... The discovery of that file sharing server's address, is always important. Frequently there is legacy methods, brittle, even broken, implementations, and obscure configuration details. They all conspire to get users tearing their hair out.

 

[Robbie]

@wblew - interesting and thoughtful post but macOS Mojave is a very old version of macOS - it is unsupported, end-of-life and will not run at all on more modern Mac hardware. There have been many networking and SMB changes in the macOS versions since, particularly after High Sierra 10.13 & Big Sur 11.1. As such, it will probably confuse the matters here.

I note your observation regarding wifi being preferred over wired ethernet. In this regard the modern systems and macOS version don't do much better. I have 10 GbE on all my desktop Macs, on my Mac server and on my main Syno NAS, yet it can still pick wifi over a physical connection. I have disabled wifi on my Mac server (only a minor inconvenience) but enabled elsewhere. It seems macOS prefers wifi when sleeping and may fail to switch-over when coming out of sleep. For hardwired Macs macOS can pick 1 GbE over 10 GbE - I disable the switch 1 GbE port at the switch so I can still enable it should the 10 GbE link fail.

Someone at Apple seems to prefer slower links!

️

 

[wblew]

I must have been mostly asleep. I should of said Monterey. My bad…

-- post merged: 10. Jun 2022 --

Agreed on the slower link being used. The other issue I have seen is having SMB’s related address discovery working correctly.

With four to choose from. NetBIOS over TCP, WINS server, AD’s Unicast DNS, and Bonjour, it can get non trivial to understand what is actually happening.

I was surprised pointing the client’s WINS setting at the Synology NAS would have such a large effect.

I should note, that all three machines; both of the Synology NASs, and my iMac are all members of an Active Director Domain, that I have setup for my home network. Despite that fact... the (current release of the) MacOS still seems to be having SMB address resolution problems.

This fact really surprised me. After all, my partner's Windows 10 workstations, also being AD members, are having absolutely no problems with Synology SMB waits and such like, as my MacOS is having.

I guess the lesson is, to understand which SMB address discovery method is being used by the current MacOS, and then to correctly configure it.

This is still a work in progress for me. My older Synology is still a problem for my Mac, so work remains.

 

[Robbie]

I must have been mostly asleep. I should of said Monterey. My bad…

-- post merged: 10. Jun 2022 --

Agreed on the slower link being used. The other issue I have seen is having SMB’s related address discovery working correctly.

With four to choose from. NetBIOS over TCP, WINS server, AD’s Unicast DNS, and Bonjour, it can get non trivial to understand what is actually happening.

I was surprised pointing the client’s WINS setting at the Synology NAS would have such a large effect.

I should note, that all three machines; both of the Synology NASs, and my iMac are all members of an Active Director Domain, that I have setup for my home network. Despite that fact... the (current release of the) MacOS still seems to be having SMB address resolution problems.

This fact really surprised me. After all, my partner's Windows 10 workstations, also being AD members, are having absolutely no problems with Synology SMB waits and such like, as my MacOS is having.

I guess the lesson is, to understand which SMB address discovery method is being used by the current MacOS, and then to correctly configure it.

This is still a work in progress for me. My older Synology is still a problem for my Mac, so work remains.

Ok, a rather more modern macOS!

My random thoughts on your (modern) setup:

1. I'd advise against your transport encryption setting - ideally should be disabled. If active your speeds can plummet plus opportunistic locking and SMB2 leases will be disabled with an impact on your SMB cache too.
2. I'm not keen on running a Synology as a Local Master Browser as there are security implications
3. Use of WINS seems at odds with disabling NetBIOS (I generally consider disabling NetBIOS as a very good thing to do)
4. The internal DNS issues should be resolved with a tweakable local DNS caching server (and gratuitous testing with dig)
5. Running an AD on a local network only adds another layer of complication that isn't warranted, especially if it is masking a internal DNS issue
6. Server DirSort should not be needed on modern Mac or Linux network, less sure about Windows but I thought modern versions no longer needed the help?
7. Never made my mind up with asynchronous read on Synology units, I have mine enabled but may disable again at some point. For whatever reason it works just fine on Xeon-equipped models but other (still capable) architectures seem to move the advantages and disadvantages around. Of course, not really an option at all on the weaker CPU units.

️

 

[wblew]

Ok, a rather more modern macOS!

My random thoughts on your (modern) setup:

1. I'd advise against your transport encryption setting - ideally should be disabled. If active your speeds can plummet plus opportunistic locking and SMB2 leases will be disabled with an impact on your SMB cache too.
2. I'm not keen on running a Synology as a Local Master Browser as there are security implications
3. Use of WINS seems at odds with disabling NetBIOS (I generally consider disabling NetBIOS as a very good thing to do)
4. The internal DNS issues should be resolved with a tweakable local DNS caching server (and gratuitous testing with dig)
5. Running an AD on a local network only adds another layer of complication that isn't warranted, especially if it is masking a internal DNS issue
6. Server DirSort should not be needed on modern Mac or Linux network, less sure about Windows but I thought modern versions no longer needed the help?
7. Never made my mind up with asynchronous read on Synology units, I have mine enabled but may disable again at some point. For whatever reason it works just fine on Xeon-equipped models but other (still capable) architectures seem to move the advantages and disadvantages around. Of course, not really an option at all on the weaker CPU units.

️

1) I had no idea there was an interaction between SMB2 leases and encryption. Good points. Disabled!


2) Me neither. Yet, my old Windows 7 machines were very unhappy without it being configured and setup.

2.1) With my last Windows 7 machine having been decommissioned, perhaps WINS is no longer needed?

2.2) Plenty of careful testing, perhaps with some Wireshark investigation for good measure, are in my future.


3) I would think so as well, and yet, a Wireshark capture this morning proved that my older Synology NAS (yet with DSM 7.1) was doing a query for the workgroup's master browser... Like, Why?


4) Definitely an area for further investigation. Work is clearly needed... after all, I want to understand why the MacOS SMB client delays are happening, not just magically tweak them out of existence. While retired now, I spent 30 years during software development. I honestly find this kind of investigation interesting...


5) Agreed. However, having an AD domain keeps my partner's Windows machines from having sporadic Synology SMB share access, and/or authentication, and/or permission issues. Hence the AD complexity, and maintenance burden is well worth the additional effort on my part.


6) Sounds good. Disabled.


7) It's a good question, isn't it? I suspect my old NAS, and even my newer NAS, see a marginal benefit at best, from having asynchronous reads enabled. In particular given;

7.1) My home network is only 1 Gbps. While my old Syno cannot keep a 1 Gbps pipe full, my new Syno hovers around 10% CPU at worst. Maybe with a 10 Gbps network, async reads might matter? Yes, a network upgrade is in my future.

Why upgrade my network? Why not. I am retired, and having fun with my home network. My lady is happy with trouble free video and music playback from our Syno NAS, and her being happy, keeps me happy. She wants the technology just work, without any problems. I am the one that enjoys tinkering with computers and networks.

7.2) There has been talk on the samba mailing list about minimal benefit from older Samba releases' async reads being enabled.

Apparently? They used a GNU libc implementation. Further, said GNU libc is apparently limited at best?

7.3) Of interest, even with DMS 7.1, the Samba and Linux kernel versions are somewhat dated. On my DS1621+, here they are:

samba -V
Version 4.10.18

uname -a
Linux crystal 4.4.180+ #42661 SMP Fri May 27 17:13:27 CST 2022 x86_64 GNU/Linux synology_v1000_1621+

On my old DS216play, the kernel is older still:

uname -a
Linux ds216play 3.10.108 #42661 SMP Fri Apr 1 15:29:25 CST 2022 armv7l GNU/Linux synology_monaco_ds216play

Cheers. Thanks for taking the time to respond. It's all food for very careful thought.

PS: Why number all points? Many years of writing, and updating, software defect reports. I found that keeping them readable was a challenge.

PPS: Here is a good SMB overview. I found it interesting reading: NAP-3 Microsoft SMB Troubleshooting

Update:

I figured out what was happening with the slow iMac => mounting of OLD Synology shares. When Time Machine is doing a backup in the background to the NEW Synology, the MacOS's foreground is delayed in continuing (frequently 500ms plus, ~1700ms one time!) its SMB transactions with the OLD Synology.

It doesn't even seem to matter whether Time Machine is backing up to an attached external HDD or over SMB to some NAS.

Basically, despite being fast Mac hardware, the MacOS SMB client seems bad at doing multiple things at once...

 

[jeyare]

RE: samba or kernel versions
OFC,
Synology has a lot of core services in old versions. They don't communicate it at all.
e.g. Samba 4.11.0 - SMB1 is disabled by default, which may be why Syno keeps smb under this version = sharing would stop working for many users. Regardless of security issues.
or
for this version you need Py min v3.4. And it is another problem.
As Python 2.7 has been End Of Life upstream since April 2020, Samba
is dropping ALL Python 2.x support in release from 4.14.
Check ver in your NAS:
python -V

The kernel version is a separate chapter for debate.

 

[Robbie]

Synology made a big deal and even produced a video on them finally bringing an update to the included SMB version (ie the one from the last decade).

They even talked about full SMB multichannel support too. Excellent news all-round I think.

Didn't happen though.

Still, nice marketing video, so there is that.

Basically, despite being fast Mac hardware, the MacOS SMB client seems bad at doing multiple things at once...

Mine are all stable and fast - including the 2 Macs I have running on 10 GbE. They have no issue communicating with other 10 GbE systems, inc Synology NASes & Windows server, or anything running at or above 100 Mbps or with any wifi clients on the same network.

As an aside, did you tick "Enable Bonjour service discovery to locate Synology NAS" and, if the network needs extra help, ticked "Enable Bonjour Time Machine broadcast via SMB" [no need to use TM, just using the extra _adisk._tcp. flag that it triggers]?

️

 

[wblew]

As an aside, did you tick "Enable Bonjour service discovery to locate Synology NAS" and, if the network needs extra help, ticked "Enable Bonjour Time Machine broadcast via SMB" [no need to use TM, just using the extra _adisk._tcp. flag that it triggers]?

Sure did. It seems my MacOS install isn't performing up to standard. It's been over 10 years since I did a clean install.

Perhaps that's a good idea.

On a different note: I got curious about the Synology installed Samba version. It's installed as 4.10.18 on my DSM 7.1. The Samba 4.10.0 release (thus 4.10 branch) was released on March 19, 2019. The currently installed 4.10.18 maintenance release was first seen on September 18, 2020. The major release is only ~3 years old, or so. It's merely a toddler, in human years.

 

[abhijitb]

Hey all

I found this thread thanks to Google, as I have had the SMB problem on macOS for quite some time (and different OS versions). For me it's that they disappear after sleep mode and can't be connected anymore - even if the devices are still pingable... If I then restart the Mac, the connections are there again perfectly and work.

My question is now if here jmd. has now made these changes that are suggested in the thread and if these have helped. Or have you made any other optimizations in the meantime?

Thank you for your input,

many greetings,

Abhijit

 

[wblew]

For me it's that they disappear after sleep mode and can't be connected anymore - even if the devices are still pingable... If I then restart the Mac, the connections are there again perfectly and work.

I have had similar issues in the past. In my case it was an expired Kerberos ticket, given that I use Active Directory to manage all SMB authentication and thus server access in my home network. For me, the Kerberos Ticket Autorenewal MacOS app nicely solved my problem.

However, your situation might be very different. I have no way to know... shrug. Good luck!

 

[abhijitb]

I have had similar issues in the past. In my case it was an expired Kerberos ticket, given that I use Active Directory to manage all SMB authentication and thus server access in my home network. For me, the Kerberos Ticket Autorenewal MacOS app nicely solved my problem.

However, your situation might be very different. I have no way to know... shrug. Good luck!

Thank you for your kind answer. Your inputs in this thread helped me a lot even we didn't have the same situation

 

[HappyDays]

I am still experiencing stalls when saving from my Mac to SMB shares on my DS420+, which are IMMEDIATELY resolved by clearing the SMB cache.

The symptom is that I try to save a file from Preview, Word, or other applications but get the revolving beach ball and the application becomes unresponsive. I then log in to my Synology box and click on "Clear SMB cache" in File services->SMB->Advanced Settings and instantly the pending save completes and the application on the Mac becomes responsive again. I see this from multiple Macs on my network.

Given how instantly the problem resolves when I clear the cache I would be surprised if there is any other issue on my network or with my hardware that is causing this issue. It has perpetuated through multiple MacOS versions, multiple revisions of DSM and the SMB package, and across various DS boxes and Macs.

Currently running macOS 13.3.1(a), DSM 7.1.1-42962 Update 5, SMB 4.15.9-0632.

Note that the latest released version of the DSM package is 4.15.13-0760 but this is only compatible with DSM 7.2 which has not been released in its final form yet. There is no hint in the release notes that this will address this issue.

Note also that the symptom does not occur for a while after clearing the cache, and so I have set a repeating task on the Synology box to restart Samba every half an hour which in general avoids the problem occurring. However if I save lots of files to the box (not a massive number, though) it will trigger and that is when I have to log in to the box and clear the cache manually.

Is no-one else experiencing this issue who finds that it is resolved as I describe? I find it hard to believe I am the only one! I cannot think of any reason this might be occurring other than some sort of a bug (or possible incompatibility issue with the macOS client) in the Samba server implementation.

 

[EAZ1964]

Had, like most MacOs users my share of SMB issues in the past.
Now for the last 2 years or so it is stable on (after 11.3).
Settings on DSM7.1 and MacOs are both default.
Something that may help me, is that the NAS is auto switched off after the backup run at 22:00. So SMB cache is emtied daily.

 

[Telos]

I am still experiencing stalls

Have you disabled SMB1

 

[HappyDays]

I have modified my nsmb.conf file to include that line, although the SMB settings in DSM have always been set to a minimum level of "SMB2 and Large MTU".

For reference, my nsmb.conf is as follows (with the protocol line just added):

[default]
signing_required=no
streams=yes
notify_off=yes
port445=no_netbios
protocol_vers_map=6
unix extensions = no
veto files = /._*/.DS_Store/

@EAZ1964 - I am pretty certain that the shut down every evening would only work if I saved very few files during the day. Even with my reboot of Samba every half an hour I still get occasions when I save a lot of files and saving stalls so that I have to log on and clear the SMB cache. I rather doubt that I am saving more files than other people as I only have two active users on the server whereas I am sure that there are many systems which support large numbers of users.

However what I don't know is what goes on inside Samba - for example, does it clear its own cache occasionally, and is the cache partitioned for different users?

 

[fredbert]

I'm guessing that's the nsmb.conf on the NAS? I don't see unix extensions or veto files when man nsmb.conf on macOS.

In DSM's Help it says using the Veto Files feature can impact performance. Maybe if you add screenshots of your File Sharing and SMB Advanced settings then we can compare them against ours. I've not noticed these issues and I have a DSM shared folder permanently mounted in Finder: it is used exactly as I used the USB drive that this replaces... create, modify, and hold media files and libraries.

I'm also on the latest DSM 7.1.1 and it's latest SMB Service. So I think the same as you. Using Ventura on main Mac and Big Sur on old MacBook. I have scheduled uploads from Mac to NAS using CCC 6, which auto-mounts other shares using SMB.

 

[Robbie]

I'm another that runs several Macs using SMB as the backbone. No issues here so also happy to compare settings. I also run a collection of Synologys on different architectures and 1 on the RC. Hopefully we can find a similar configuration to yours.

️