DSM 7.2 Add a GPU for transcoding

[m4v3r1ck]

Great write-up, thanks for sharing.

1. Is anybody willing to post some pictures on how to apply - best practice - the Kapton Tape to the riser and the GPU?
2. What width in mm is the best for the Kapton tape to be used for this fix?

Much appreciated!

Cheers

 

[Silversurfer79]

Im having this bracket printed for the T400, I'll send image once I have it done.

T400 best 3D printing models・1 free models to download・Cults

Discover 3D models for 3D printing related to T400. Download your favorite STL files and make them with your 3D printer. Have a good time!

cults3d.com

On another note, any updates to the latest version of DSM with any issues?

 

[m4v3r1ck]

Im having this bracket printed for the T400, I'll send image once I have it done.

T400 best 3D printing models・1 free models to download・Cults

Discover 3D models for 3D printing related to T400. Download your favorite STL files and make them with your 3D printer. Have a good time!

cults3d.com

On another note, any updates to the latest version of DSM with any issues?

Looks very good, thanks for sharing.

Are the dimensions - especially the height - of this 3D printed bracket adjusted to the total new height of the T400 inserted into the riser, so the bracket can be screwed into the DS1821+ PCI slot, to secure the T400 with it being inserted into the PCIe riserboard x8 > x16?

Excited to see your images of the end result put together.

Cheers

 

[m4v3r1ck]

Hi all,

Did some research and got some great support/advice on the Dutch equivalent of this forum and was noted (warned) that the Max Power Draw of the PCIe 3.0 x8 slot is limited to 25! Watt. Do you guys use some auxiliary power to overcome the 25 Watt power draw limit of the used slot in the DS1821+?

Thanks in advance for your further elaboration on this subject.

Cheers

 

[m4v3r1ck]

Additional info YouTube video, for those wondering about the build and using Kapton Tape.

At your Service!

 

[m4v3r1ck]

My SynologY Technical Support ticket about the max power draw of the PCIe 3.0 x8 slot is answered as followed. Any electrical engineers here on the forum, whom are willing tot disclose the correct info, as for now I'm not biting the bullet. Abort mission...

NO DISCLOSING INFORMATION GIVEN BY SYNOLOGY, AREA 51?

 

[Coop777]

That answer shouldn't surprise anyone, especially with unsupported 3rd party hardware....

 

[Silversurfer79]

Can anyone help please! Im at my wits end with his now, I think I have a bug.

I have reinstalled both NVIDIA drivers and Simple Permission Manager, all the version are now at the latest and still I have a issue.

DSM 7.3.2-86009 Update 1

I can transcode in Plex, It works perfectly... So I dont get it... see below.

But I cant see the information here, though it shows the clot as occupied.

Or here,

My permissions look good, see below.

What am I missing?

 

[Silversurfer79]

I also found when I add the below into a Stack in Portainer

devices:
- /dev/dri:/dev/dri

to the Stack in Portainer, the container fails with, this suggest the (confirm render node exists)

But if I "comment" it out the UI loads and works perfectly, the log makes mention to only using CPU as there is no GPU, go figure...

Also another error when I try find the below directory, Grok says it should exist.

sudo ls -l /dev/dri/
"ls: cannot access '/dev/dri/': No such file or directory"

 

[tmkk]

I may report that I successfully installed and run PNY Quadro RTX 2000E ADA 16GB GDDR6. Although a little expensive this gpu is very suitable (16GB VRAM) for self hosted LLM's. This gpu takes only one slot and can be fully powered by the pcie (needs only 50W).

 

[tmkk]

You should put following content in the compose.yml to utilize nvidia gpu in docker

 

[Silversurfer79]

Thanks @tmkk,

What about devices, is it present?

 

[tmkk]

Thanks @tmkk,

What about devices, is it present?
View attachment 33833

This statement is required in compose.yml to utilize integrated gpu present in some intel cpu's like intel celeron J4125 (popular cpu used in numerous synology units like ds920+, etc.).

 

[DSfuchs]

Many thanks to all for the great guide; I also managed to get my T400 graphics card up and running on my RS3621xs+ (broadwellnk platform) using DSM 7.4.

After all, Synology refuses to provide any support for the GPU or its usage, even with DSM 7.4!

So, I’d like to contribute my download script for a suitable AI "gemma-2-2b-it-abliterated" model for the T400 I use.

• 4-bit using T400 with 2GB VRAM: gemma-2-2b-it-abliterated-Q4_k_m.gguf
• 8-bit using T400 with 4GB VRAM: gemma-2-2b-it-abliterated-Q8_0.gguf

download_gguf.sh:

#!/bin/bash
# ==========================================================
# --- Simply enter your preferred details here.          ---
# ==========================================================
DEVELOPER="bartowski"
REPO_NAME="gemma-2-2b-it-abliterated-GGUF"
FILE_NAME="gemma-2-2b-it-abliterated-Q4_k_m.gguf"
TARGET_DIR="gemma-2-2b-it-abliterated"

# ==========================================================
# --- AUTOMATIC HIGH-SPEED INJECTOR (KEEP HANDS OFF)     ---
# ==========================================================

# Create the target directory and switch into it.
mkdir -p "$TARGET_DIR"
cd "$TARGET_DIR" || exit 1

echo "=========================================================="
echo "???  ACTIVATE UNIVERSAL GGUF INJECTOR ON DSM"
echo "Entwickler:  $DEVELOPER"
echo "Repository:  $REPO_NAME"
echo "Lade Datei:  $FILE_NAME"
echo "Zielordner:  $(pwd)"
echo "=========================================================="

# The rock-solid formula that I just verified live!
DOWNLOAD_URL="https://huggingface.co/${DEVELOPER}/${REPO_NAME}/resolve/main/${FILE_NAME}"

echo "Signal to Hugging Face is green"

The rock-solid formula that I just verified live!..."
echo "URL: $DOWNLOAD_URL"
echo "----------------------------------------------------------"

# The foolproof cURL command with automatic resumption after interruption
curl -L -C - -o "$FILE_NAME" "$DOWNLOAD_URL"

echo ""
echo "=========================================================="
echo "?? Finish line reached! File is on the NAS, error-free...."
echo "=========================================================="

Modelfile:

FROM /models/gemma-2-2b-it-abliterated/gemma-2-2b-it-abliterated-Q8_0.gguf
TEMPLATE "{{ if .System }}<start_of_turn>system\n{{ .System }}<end_of_turn>\n{{ end }}{{ if .Prompt }}<start_of_turn>user\n{{ .Prompt }}<end_of_turn>\n{{ end }}<start_of_turn>model\n{{ .Response }}<end_of_turn>\n"

# We are keeping the context set to the stable 1024 so that Ollama doesn't block.
PARAMETER num_ctx 1024

# NOW, TURNING UP THE BOOST:
# We’re increasing the batch size for prompt processing from 512 to 1024!
# This loads more tokens into the CUDA cores simultaneously and fills the vRAM with pure performance!
PARAMETER num_batch 1024

compose.yaml:

version: "3.8"

services:
  # --- BACKEND 1: LOCALAI (with the correct CUDA-12 identifier) ---
  localai:
    container_name: localai_t400
    image: localai/localai:latest-gpu-nvidia-cuda-12
    restart: always
    ports:
      - "8080:8080"
    environment:
      - MODELS_PATH=/models
      - DEBUG=true
    volumes:
      - /volume1/docker/localai/models:/models
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

  # --- BACKEND 2: OLLAMA ---
  ollama:
    container_name: ollama
    image: ollama/ollama:latest
    restart: always
    ports:
      - "11434:11434"
    # INSTALL THE TURBO-BOOST HERE:
    environment:
      - OLLAMA_NUM_PARALLEL=4
    volumes:
      - /volume1/docker/ollama:/root/.ollama
      # Here, Ollama accesses the same folder!
      - /volume1/docker/localai/models:/models
      # Here is the direct link to a text Maildir:
      # - /volume1/docker/localai/Maildir:/mails
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

  open-webui:
    container_name: open-webui
    image: ghcr.io/open-webui/open-webui:main
    restart: always
    ports:
      - "3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes:
      - /volume1/docker/open-webui:/app/backend/data

For example, I have made my email directory containing over 40,000 entries available to Gemma so that she can answer any question related to me.