updated version of Fine-Tune RF-DETR on Object Detection Dataset notebook with instructions for hyperparameters tuning and multi-GPU training

Author: SkalskiP

notebooks/how-to-finetune-rf-detr-on-detection-dataset.ipynb

@@ -771,7 +771,7 @@
       "metadata": {
         "id": "_dGO4a7eTbFX"
       },
-      "execution_count": 1,
+      "execution_count": null,
       "outputs": []
     },
     {
@@ -797,7 +797,7 @@
         "id": "ms0ps6ZCT2xs",
         "outputId": "6a5ffafe-73c6-45f9-f16b-36e400cc2ded"
       },
-      "execution_count": 2,
+      "execution_count": null,
       "outputs": [
         {
           "output_type": "stream",
@@ -865,7 +865,7 @@
       "metadata": {
         "id": "8DCCwexcU6gO"
       },
-      "execution_count": 4,
+      "execution_count": null,
       "outputs": []
     },
     {
@@ -949,7 +949,7 @@
         "id": "PtOufRspVekp",
         "outputId": "711403bd-2543-4582-de1b-1e6b369045af"
       },
-      "execution_count": 5,
+      "execution_count": null,
       "outputs": [
         {
           "output_type": "stream",
@@ -1037,7 +1037,7 @@
         "id": "hQkMUyB0lROT",
         "outputId": "01fc243a-7911-4762-f44b-4652f748e138"
       },
-      "execution_count": 6,
+      "execution_count": null,
       "outputs": [
         {
           "output_type": "stream",
@@ -1074,7 +1074,24 @@
     {
       "cell_type": "markdown",
       "source": [
-        "## Train RF-DETR on custom dataset"
+        "## Train RF-DETR on custom dataset\n",
+        "\n",
+        "### Choose the right `batch_size`\n",
+        "\n",
+        "Different GPUs have different amounts of VRAM (video memory), which limits how much data they can handle at once during training. To make training work well on any machine, you can adjust two settings: `batch_size` and `grad_accum_steps`. These control how many samples are processed at a time. The key is to keep their product equal to 16 — that’s our recommended total batch size. For example, on powerful GPUs like the A100, set `batch_size=16` and `grad_accum_steps=1`. On smaller GPUs like the T4, use `batch_size=4` and `grad_accum_steps=4`. We use a method called gradient accumulation, which lets the model simulate training with a larger batch size by gradually collecting updates before adjusting the weights.\n",
+        "\n",
+        "### Train with multiple GPUs\n",
+        "\n",
+        "You can fine-tune RF-DETR on multiple GPUs using PyTorch’s Distributed Data Parallel (DDP). Create a `main.py` script that initializes your model and calls `.train()` as usual than run it in terminal.\n",
+        "\n",
+        "```bash\n",
+        "python -m torch.distributed.launch \\\n",
+        "    --nproc_per_node=8 \\\n",
+        "    --use_env \\\n",
+        "    main.py\n",
+        "```\n",
+        "\n",
+        "Replace `8` in the `--nproc_per_node argument` with the number of GPUs you want to use. This approach creates one training process per GPU and splits the workload automatically. Note that your effective batch size is multiplied by the number of GPUs, so you may need to adjust your `batch_size` and `grad_accum_steps` to maintain the same overall batch size."
       ],
       "metadata": {
         "id": "vmT8f_bAq3zX"
@@ -1143,7 +1160,7 @@
         "id": "7UEX3GVCmYaq",
         "outputId": "a579ba09-43ab-4863-d2b6-94dbd77bbb6b"
       },
-      "execution_count": 8,
+      "execution_count": null,
       "outputs": [
         {
           "output_type": "display_data",
@@ -1203,7 +1220,7 @@
         "id": "gcjxmZeqqAdv",
         "outputId": "356c1fea-316e-4b3d-888c-fe869e8adffd"
       },
-      "execution_count": 9,
+      "execution_count": null,
       "outputs": [
         {
           "output_type": "display_data",
@@ -1249,7 +1266,7 @@
       "metadata": {
         "id": "xm-lmRWLswO4"
       },
-      "execution_count": 10,
+      "execution_count": null,
       "outputs": []
     },
     {
@@ -1304,7 +1321,7 @@
         "id": "msor_5HgAkm3",
         "outputId": "3c43efc2-c7d8-42b8-8d6d-c235f537222a"
       },
-      "execution_count": 11,
+      "execution_count": null,
       "outputs": [
         {
           "output_type": "display_data",
@@ -1365,7 +1382,7 @@
         "id": "RFEgIOz1YDCe",
         "outputId": "8f885c5c-5122-44a6-d8ed-3783e442cc69"
       },
-      "execution_count": 14,
+      "execution_count": null,
       "outputs": [
         {
           "output_type": "display_data",
@@ -1412,7 +1429,7 @@
         "id": "szxs3PZsBVxa",
         "outputId": "46b8ab4b-41a0-4fc3-ff63-bb28bad0b442"
       },
-      "execution_count": 20,
+      "execution_count": null,
       "outputs": [
         {
           "output_type": "stream",
@@ -1439,7 +1456,7 @@
         "id": "fxqvXOQcsRF2",
         "outputId": "240e349f-4051-4c7a-eb74-2c521ac03e2b"
       },
-      "execution_count": 21,
+      "execution_count": null,
       "outputs": [
         {
           "output_type": "display_data",
@@ -1472,7 +1489,7 @@
         "id": "WuiNB-UM1xsJ",
         "outputId": "822fea57-acb9-4d09-ad34-02aa640ad808"
       },
-      "execution_count": 24,
+      "execution_count": null,
       "outputs": [
         {
           "output_type": "display_data",