# Finetuned FunctionGemma Model: `emredeveloper/functiongemma-tools`

This repository hosts a finetuned version of Google's `functiongemma-270m-it` model, specifically adapted for advanced tool-calling capabilities with integrated reasoning. This model has been enhanced using the `LLM360/TxT360-3efforts` dataset, which focuses on providing detailed thinking processes alongside tool calls.

## Model Overview

- **Base Model**: `unsloth/functiongemma-270m-it`
- **Finetuning Method**: LoRA (Low-Rank Adaptation) using Unsloth library.
- **Dataset**: `LLM360/TxT360-3efforts` (agent split, 50,000 examples streamed).
- **Key Enhancement**: Improved tool-calling with explicit `<think>...</think>` blocks for internal reasoning, crucial for complex multi-step tasks.

## Finetuning Details

This model was finetuned in Google Colab using the Unsloth library, which significantly speeds up the finetuning process and reduces VRAM usage.

### Training Configuration:

- **Max Sequence Length**: 4096
- **LoRA Rank (r)**: 128
- **LoRA Alpha**: 256
- **LoRA Dropout**: 0
- **Gradient Checkpointing**: Enabled with "unsloth" optimization
- **Batch Size**: 4 (per device), with gradient accumulation steps = 2
- **Learning Rate**: 2e-4
- **Optimizer**: `adamw_8bit`
- **Training Steps**: 100

Special attention was given to applying a custom chat template that incorporates `<think>` tags for explicit reasoning and aligns with the `functiongemma`'s tool-calling format. The training focused on responses only, masking out the instruction part to enhance the model's generation quality.

## How to Use

This model can be loaded and used for inference with the Hugging Face `transformers` library, especially when combined with Unsloth for optimized performance.

### Installation

First, ensure you have the necessary libraries installed:

```bash
pip install transformers unsloth[cuda] torch
```

### Loading the Model

```python
from unsloth import FastLanguageModel
import torch
from transformers import TextStreamer

max_seq_length = 4096

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "emredeveloper/functiongemma-tools", # Your finetuned model on Hugging Face Hub
    max_seq_length = max_seq_length,
    load_in_4bit = False,
    load_in_8bit = False,
    load_in_16bit = True, # Or False if you saved merged_4bit
)
```

### Inference Example

To perform inference, you'll need to construct your messages and tools according to the `functiongemma` chat template. Here's an example demonstrating basic tool calling with thinking:

```python
import json

tools_example = [
    {
        "type": "function",
        "function": {
            "name": "get_amazon_product_details",
            "description": (
                "Retrieves comprehensive product information from Amazon, "
                "including title, price, description, specifications, and availability."
            ),
            "parameters": {
                "type": "object",
                "properties": {
                    "asin": {
                        "type": "string",
                        "description": "The Amazon ASIN of the product.",
                    }
                },
                "required": ["asin"],
            },
        },
    }
]

messages_example = [
    {
        "role": "system",
        "content": (
            "You are a shopping assistant. Use tools when you need detailed "
            "Amazon product data such as price and specifications."
        ),
    },
    {
        "role": "user",
        "content": "Is the espresso machine with ASIN B0XYZ12345 any good for home use?"
    },
]

# Apply the chat template for generation
text = tokenizer.apply_chat_template(
    messages_example,
    tools = tools_example,
    tokenize = False,
    add_generation_prompt = True,
).removeprefix('<bos>') # Remove <bos> token if present

# Generate a response
_ = model.generate(
    **tokenizer(text, return_tensors = "pt").to("cuda"),
    max_new_tokens = 1024,
    streamer = TextStreamer(tokenizer, skip_prompt = False),
    top_p = 0.95, top_k = 64, temperature = 1.0,
)
```

This will produce an output similar to:

```
<start_of_turn>model
<think>User is asking for an opinion, but I need factual product details first such as price, features, and reviews. I should call the Amazon product details tool with the provided ASIN.</think><start_function_call>call:get_amazon_product_details{asin:<escape>B0XYZ12345<escape>}<end_function_call>
```

## Applications

This finetuned `functiongemma` model is ideal for:

- **Advanced AI Assistants**: Building intelligent agents that can reason about complex tasks and use external tools effectively.
- **Tool-Augmented LLMs**: Enhancing LLMs with the ability to dynamically call functions and interpret their results.
- **Complex Workflow Automation**: Automating multi-step processes that require logical reasoning and interaction with external systems.
- **Research in Tool Learning**: Studying and developing more sophisticated tool-learning mechanisms for LLMs.

## Feedback and Issues

For any questions, issues, or contributions, please refer to the Unsloth Discord channel or open an issue on the Hugging Face repository.