Two mandatory prerequisites for VM in Syno NASs:
... 4GB+ RAM
... specific models only
Docker - no such limitations.
There is no question which is better (VM vs Docker), because the usage of them is driven by purposes. They both are similar from isolated environment point of view.
I found really clear description for you, found in web (lazy to write):
Docker is container based technology and containers are just user space of the operating system. At the low level, a container is just a set of processes that are isolated from the rest of the system, running from a distinct image that provides all files necessary to support the processes. It is built for running applications. In Docker, the containers running share the host OS kernel.
A Virtual Machine, on the other hand, is not based on container technology. They are made up of user space plus kernel space of an operating system. Under VMs, server hardware is virtualized. Each VM has Operating system (OS) & apps. It shares hardware resource from the host.
VMs & Docker – each comes with benefits and demerits. Under a VM environment, each workload needs a complete OS. But with a container environment, multiple workloads can run with 1 OS. The bigger the OS footprint, the more environment benefits from containers. With this, it brings further benefits like Reduced IT management resources, reduced size of snapshots, quicker spinning up apps, reduced & simplified security updates, less code to transfer, migrate and upload workloads.
However, containers are typically much smaller and faster, which makes them a much better fit for fast development cycles and microservices. The trade-off is that containers don’t do true virtualization; you can’t run a windows container on a Linux host for example.