Intent Discovery Tool
This tool helps developers identify and extract intents from large datasets of user queries, which is essential for building effective conversational AI models. It's useful for discovering hidden patterns and relationships in user input, enabling more accurate intent recognition and improved chatbot performance. The tool utilizes techniques from natural language processing and machine learning to uncover underlying intents.
Installation
To use this tool, you need to install the required packages. You can do this by running the following command:
pip install spacy scikit-learn
python -m spacy download en_core_web_smUsage
To use this tool, simply run the following command:
python intent_discovery.py --input_file input.txt --output_file output.jsonReplace input.txt with the path to your input file and output.json with the path to your desired output file.
Source Code
import argparse
import json
import spacy
from sklearn.cluster import KMeans
from sklearn.feature_extraction.text import TfidfVectorizer
try:
spacy.load('en_core_web_sm')
except OSError:
print("Downloading language model for the spaCy 'en_core_web_sm'")
spacy.cli.download("en_core_web_sm")
def load_data(input_file):
try:
with open(input_file, 'r') as f:
queries = [line.strip() for line in f.readlines()]
return queries
except FileNotFoundError:
print(f"File {input_file} not found.")
return []
def extract_intents(queries):
nlp = spacy.load('en_core_web_sm')
vectorizer = TfidfVectorizer()
if not queries:
return {}
vectors = vectorizer.fit_transform(queries)
kmeans = KMeans(n_clusters=min(5, len(queries)))
kmeans.fit(vectors)
labels = kmeans.labels_
intents = {}
for i, label in enumerate(labels):
if label not in intents:
intents[label] = []
intents[label].append(queries[i])
return intents
def extract_keywords(intents):
nlp = spacy.load('en_core_web_sm')
keywords = {}
for label, queries in intents.items():
doc = nlp(' '.join(queries))
keywords[label] = [token.text for token in doc if token.pos_ == 'NOUN']
return keywords
def main(input_file, output_file):
queries = load_data(input_file)
intents = extract_intents(queries)
keywords = extract_keywords(intents)
with open(output_file, 'w') as f:
json.dump(keywords, f)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Intent Discovery Tool')
parser.add_argument('--input_file', required=True, help='Input file containing user queries')
parser.add_argument('--output_file', required=True, help='Output file for discovered intents and keywords')
args = parser.parse_args()
main(args.input_file, args.output_file)README
Intent Discovery Tool
This tool helps developers identify and extract intents from large datasets of user queries, which is essential for building effective conversational AI models.
Installation
To use this tool, you need to install the required packages. You can do this by running the following command:
pip install spacy scikit-learn
python -m spacy download en_core_web_smUsage
To use this tool, simply run the following command:
python intent_discovery.py --input_file input.txt --output_file output.jsonReplace input.txt with the path to your input file and output.json with the path to your desired output file.
Input File Format
The input file should contain one query per line.
Output File Format
The output file will contain a JSON object with the discovered intents and keywords.
Community
Downloads
···
Rate this tool
No ratings yet — be the first!
Details
- Tool Name
- intent_discovery
- Category
- Conversational AI
- Generated
- August 9, 2026
- Tests
- Passing ✅
- Fix Loops
- 2
Quick Install
Clone just this tool:
git clone --depth 1 --filter=blob:none --sparse \ https://github.com/ptulin/autoaiforge.git cd autoaiforge git sparse-checkout set generated_tools/2026-08-09/intent_discovery cd generated_tools/2026-08-09/intent_discovery pip install -r requirements.txt 2>/dev/null || true python intent_discovery.py