Tool calling with Moonshot's Kimi K2 mode

Moonshot's new https://platform.moonshot.ai/console/account

The direct API pricing from Moonshot AI is approximately:

Input Tokens: ~$0.60 per 1 million tokens

Output Tokens: ~$2.50 per 1 million tokens

Several inference providers in the USA are now supporting Kimi 2 also. Kim 2 is a mixture of expert model with 32 billion parameters active at any time during inference. The total model size is a trillion parameters.

Tool use using the OpenAI Python library is fairly straightforward:

from typing import *
import os
import json
from openai import OpenAI
from openai.types.chat.chat_completion import Choice
client = OpenAI(
base_url="https://api.moonshot.ai/v1",
api_key=os.environ.get("MOONSHOT_API_KEY"),
)

# The specific implementation of the search tool
def search_impl(arguments: Dict[str, Any]) -> Any:
"""
When using the search tool provided by Moonshot AI, you just need to return the arguments as they are,
without any additional processing logic.
"""
return arguments

# START: Added Tool Implementations
# ---------------------------------
def list_files_impl() -> str:
"""Lists files in the current directory, one per line."""
print("--- Calling list_files_impl ---")
try:
files = os.listdir('.')
if not files:
return "The current directory is empty."
return "\n".join(files)
except Exception as e:
return f"Error listing files: {e}"

def get_file_contents_impl(filename: str) -> str:
"""Reads and returns the contents of a file."""
print(f"--- Calling get_file_contents_impl for file: {filename} ---")
try:
with open(filename, 'r', encoding='utf-8') as f:
return f.read()
except FileNotFoundError:
return f"Error: File '{filename}' not found."
except Exception as e:
return f"Error reading file '{filename}': {e}"
# -------------------------------
# END: Added Tool Implementations


def chat(messages) -> Choice:
completion = client.chat.completions.create(
model="moonshot-v1-8k", # Using a standard model for tool use demonstration
messages=messages,
temperature=0.3,
tools=[
# Tool 1: Moonshot's built-in web search
{
"type": "builtin_function",
"function": {
"name": "$web_search",
},
},
# START: Added Tool Schemas
# -------------------------
# Tool 2: List files in current directory
{
"type": "function",
"function": {
"name": "list_files",
"description": "Lists all files in the current working directory, one file per line.",
"parameters": {
"type": "object",
"properties": {},
},
},
},
# Tool 3: Get contents of a file
{
"type": "function",
"function": {
"name": "get_file_contents",
"description": "Returns the full contents of a specified file in the current directory.",
"parameters": {
"type": "object",
"properties": {
"filename": {
"type": "string",
"description": "The name of the file whose contents are to be retrieved."
}
},
"required": ["filename"],
},
},
}
]
)
return completion.choices[0]
def main():
messages = [
{"role": "system", "content": "You are Kimi, an AI assistant who can browse the web and interact with the local file system. Return all answers in English."},
]
# Updated question to demonstrate file system tools
messages.append({
"role": "user",
"content": "Please list the files in the current directory, and then show me the contents of the python script you find."
##"content": "Please search for Mark Watson AI consultant and author, and tell me what his hobbies are."
})
finish_reason = None
while finish_reason is None or finish_reason == "tool_calls":
choice = chat(messages)
finish_reason = choice.finish_reason
if finish_reason == "tool_calls":
messages.append(choice.message)
for tool_call in choice.message.tool_calls:
tool_call_name = tool_call.function.name
# Note: Arguments can be an empty string if no parameters are needed.
tool_call_arguments_str = tool_call.function.arguments
tool_call_arguments = {}
if tool_call_arguments_str and tool_call_arguments_str.strip():
tool_call_arguments = json.loads(tool_call_arguments_str)

print(f"Model wants to call tool '{tool_call_name}' with arguments: {tool_call_arguments}")
# START: Updated Tool Handling Logic
# ----------------------------------
if tool_call_name == "$web_search":
tool_result = search_impl(tool_call_arguments)
elif tool_call_name == "list_files":
tool_result = list_files_impl()
elif tool_call_name == "get_file_contents":
filename = tool_call_arguments.get('filename')
if filename:
tool_result = get_file_contents_impl(filename=filename)
else:
tool_result = "Error: 'filename' parameter is required."
else:
tool_result = f"Error: unable to find tool by name '{tool_call_name}'"
# --------------------------------
# END: Updated Tool Handling Logic
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"name": tool_call_name,
"content": json.dumps({"result": tool_result}) # Encapsulate result in a JSON object for consistency
})
print("\n--- Final Answer ---\n")
print(choice.message.content)
if __name__ == '__main__':
main()




Popular posts from this blog

I am moving back to the Google platform, less excited by what Apple is offering

I have been been playing with the Apple Intelligence beta’s in iPadOS and macOS and while I like the direction Apple is heading I am getting more use from Google’s Gemini, both for general analysis of very large input contexts, as well as effective integration my content in Gmail, Google Calendar, and Google Docs. While I find the latest Pixel phone to be compelling, I will stick with Apple hardware since I don’t want to take the time to move my data and general workflow to a Pixel phone. The iPhone is the strongest lock-in that Apple has on me because of the time investment to change. The main reason I am feeling less interested in the Apple ecosystem and platform is that I believe that our present day work flows are intimately wrapped up with the effective use of LLMs, and it is crazy to limit oneself to just one or two vendors. I rely on running local models on Ollama, super fast APIs from Groq (I love Groq for running most of the better open weight models), and other APIs from Mist...
Read more

AI update: The new Deepseek-R1 reasoning language model, Bytedance's Trae IDE, and my new book

 I spent a few days experimenting with Cursor last week. Bytedance's Trae IDE is very similar and is currently free to use with Claude Sonnet 3.5 and GPT-4o:  https://www.trae.ai/home  I would like to use Trae with my own API accounts but currently Bytedance is paying for LLM costs. I have been experimenting with the qwen2.5 and qwen2.5-coder models that easily run on my M2Pro 32G Mac. For reasoning I have been going back to using OpenAI O1 and Claude Sonnet, but after my preliminary tests with Deepseek-R1, I feel like I can do most everything now on my personal computer. I am using:  ollama run deepseek-r1:32b I recently published my new book “ Ollama in Action: Building Safe, Private AI with LLMs, Function Calling and Agents ” that can be read free online at  https://leanpub.com/ollama/read
Read more

Getting closer to AGI? Google's NoteBookLM and Replit's AI Coding Agent

Putting "closer to AGI?" in a blog title might border on being clickbait, but I will argue that it is not! I have mostly earned my living in the field of AI since 1982 and I argue that the existence of better AI driven products and the accelerating rate of progress in research, that we are raising the bar on what we consider AGI to be. I have had my mind blown twice in the last week: Today I took the PDF for my book "Practical Artificial Intelligence Programming With Clojure ( you can read it free online here ) and used it to create a notebook in Google's  NotebookLM  and asked for a generated 8 minute podcast. This experimental app created a podcast with two people discussing my book accurately and showing wonderful knowledge of technology. If you want to listen to the audio track that Google's NotebookLM created,  here is a link to the WAV audio file Last week I signed up for a one year plan on  Replit.com  after trying the web based IDE for Haskell and Python...
Read more