How-To Topics
Best Practices
This section provides essential recommendations for building effective Search AI solutions across three critical areas: data ingestion, chunking strategy, and LLM selection.1. Data Ingestion
The quality of your data directly impacts search performance. Ensure you ingest the right content in the right format.Supported Data Sources
Search AI accepts content from:- Files: PDF, DOCX, PPTX, TXT, XLSX
- Websites: Web pages and HTML content
- Connectors: Third-party applications
File Best Practices
Document Quality- Use digitally created documents rather than scanned or handwritten files
- Maintain consistent layouts across pages
- Ensure documents are unencrypted and password-free
- Keep content concise and well-structured for better search accuracy
- Remove unnecessary headers, footers, and metadata
- Single-column documents work best
- Multi-column layouts may reduce accuracy and need tuning
- Use clear section headers and logical organization
- Avoid switching between single and multi-column formats in the same document
- Compressed PDFs (can cause distortion)
- Multi-page tables (hard to process)
- Scanned or heavily formatted files
- Inconsistent formatting
- Provide text descriptions for key information in images
- Add contextual summaries before or after tables and images
- Use meaningful titles for images and tables
- Update extraction strategies for documents with significant visual content
Website Best Practices
Structure- Follow schema.org standards for metadata
- If not using schema.org, apply consistent heading logic (h1, h2, p tags)
- Standard HTML tags provide the best results
- Custom CSS structures may need fine-tuning
- Override default processing using Document Workbench for custom layouts
- Define custom extraction rules for non-standard structures
Connector Integration Best Practices
Relevance and Filtering- Ingest only relevant data for your use case
- Use advanced filters to select valuable content
- Avoid pulling entire datasets to prevent noise
- Limit ingestion frequency to avoid system overload
- Monitor logs and adjust based on performance
- Use incremental updates instead of full re-ingestion
2. Chunking Strategy
Chunking breaks content into smaller pieces for better search and retrieval. Choose your chunk size based on your specific needs. Default chunk size: 1000 tokens (customize based on your use case)When to Use Smaller Chunks (300-500 tokens)
- Precise question answering: When answers are in short text segments
- Technical documentation: Dense content with tightly packed concepts
- Multiple topics: Documents covering various subjects requiring targeted retrieval
- Limited context windows: LLMs with smaller capacity
- Memory efficiency: Optimizing storage and processing
- Cost sensitivity: Managing token usage
When to Use Larger Chunks (1000+ tokens)
- Reasoning tasks: When context and relationships between concepts matter
- Narrative content: Stories, case studies, or arguments that need coherence
- Contextual dependency: Information requiring surrounding text
- Cross-paragraph references: Content with internal references
- Multi-step procedures: Processes that must be followed in sequence
- Conceptual understanding: When grasping themes is more important than specific facts
3. LLM Selection and Configuration
Your choice of LLM shapes performance, accuracy, cost, and user experience.Model Selection
Cost-Sensitive Applications- Recommended: GPT-4o mini
- Best for: High query volume, straightforward retrieval, budget constraints
- Trade-offs: May struggle with complex reasoning but offers faster responses
- Recommended: GPT-4o or similar high-performance models
- Best for: Customer-facing apps, complex documents, technical/medical/legal content
- Trade-offs: Higher costs but better accuracy and coherence
Context Window Considerations
The context window is the amount of text a model can process in one call. This is critical for RAG applications. Matching Window Size to Chunks- Small chunks (300-500 tokens): 8k-16k context windows are sufficient
- Can accommodate 15-40 chunks
- Examples: GPT-3.5 Turbo (16k)
- Medium chunks (1k-2k tokens): 16k-32k context windows recommended
- Can accommodate 8-15 chunks
- Large chunks (3k-5k tokens): 32k-128k context windows essential
- Examples: GPT-4o (128k), Claude 3 Opus (200k)
Token Management
Your context window accommodates three parts:- System and User Prompts (500-1000 tokens) — Instructions, format specs, custom domain instructions
- Retrieved Chunks (70-90% of total usage) — Varies by chunk size, number of chunks, and search settings
- Model Response (500-2000 tokens) — Controlled through output length instructions
Example Token Calculations
Example 1: 16k Context Window
- System prompt: 500 tokens
- Chunks: 12 × 1000 = 12,000 tokens
- Response: 1,500 tokens
- Total: 14,000 tokens ✓ Fits in 16k window
- System prompt: 800 tokens
- Chunks: 20 × 800 = 16,000 tokens
- Response: 1,200 tokens
- Total: 18,000 tokens ✗ Exceeds 16k window → Error
Cost Optimization
Optimization Tips
- For verbose responses, reduce “Max tokens for Chunks”
- For complex prompts, account for their increased token usage
- Monitor usage patterns and adjust accordingly
- Use lower temperature settings (0.0-0.3) for factual responses
Custom LLM Implementation
When using custom or third-party LLMs:- Search AI doesn’t auto-detect context window limits for custom LLMs
- Set maximum input token limits manually
- Configure “Max tokens for chunks” according to context window
- Test regularly to prevent overflow errors
- Adjust temperature settings for your use case (lower is better for factual content)
Frequently Asked Questions
Application Training
Training prepares ingested content for search by applying configurations, extracting chunks, and generating embeddings. Training is required whenever content or configuration changes.Training Types
Automatic Training
The application automatically trains when new content is ingested through file uploads, web crawls, or connectors.
When auto-training initiates, a banner appears at the top of the application interface.
Manual Training
Use the Train button on the Extract or Vector Configuration page to force retraining. When Manual Training is Required:
Training Scope Based on Change Type:
Example: If a new extraction strategy is introduced for a particular document, the application is trained only for that document without affecting the chunks related to other content. If, however, the embedding model is updated or embedding fields are modified, new embeddings are generated for all the content.
Training Logs
View detailed training logs with document-level visibility:- Navigate to the Extract page
- Click the dropdown with the Train option
- Select View Training Logs
Click individual records to view details grouped by extraction strategy.
Important Notes
- Manual chunk edits are overwritten during retraining for affected content only.
- Manually resynchronizing a connector may require manual training trigger (known issue).
Multilingual Support
Search AI supports multilingual capabilities, enabling users to interact in their preferred language.Core Capabilities
Key Highlights
- 100+ Languages Supported for indexing, querying, and answer generation.
- Works with any language supported by your chosen LLM and vector generation model, using the Text Extraction strategy and Vector Retrieval method.
- No additional configuration required.
Language Support Requirements
Multilingual support works with any language supported by your configured LLM and vector generation model when using:- Text Extraction strategy
- Vector Retrieval method
Widely Supported Languages
Search AI supports languages commonly handled by advanced LLMs and embedding models like BGE-M3. Refer to your LLM or vector generation model’s official documentation for a comprehensive list.Language-Specific Extraction Capabilities
Language-Specific Retrieval Strategy Support
- English: Vector Retrieval and Hybrid Retrieval
- Non-English: Vector Retrieval
Supported Answer Generation Models
Answer generation quality depends on the language capabilities of the underlying LLM. Please refer to the official list of supported languages from the LLM provider.Recommendations
To optimize multilingual performance:- Choose the right LLM - Select models with strong support for your target languages. Refer to the official list of languages supported by the LLM.
- Customize prompts - Create language-specific prompts to improve answer quality and relevance.
- Test performance - Evaluate different LLMs for your specific use case in your target language.
- Monitor quality - Regularly assess answer quality across languages and adjust configurations as needed.
Best Practices
- Verify model compatibility — Ensure your LLM and embedding model support target languages
- Test across languages — Validate answer quality in each supported language
- Consider re-ranker selection — Choose based on primary language requirements
- Monitor performance — Low-resource languages may have reduced accuracy
User Feedback Handling
The feedback mechanism allows end users to rate response quality, helping evaluate and improve answer delivery.How Feedback Works
Users express satisfaction through thumbs up/down actions captured via:- Web SDK
- Public API
Enabling Feedback
- Navigate to Answer Generation
- Enable Feedback Configuration
Capturing Feedback
Via Web SDK: When enabled, thumbs up/down icons appear with each answer in the SDK interface. Via API: Use the Feedback API to capture feedback programmatically.Viewing Feedback Data
Feedback appears in Analytics > Search AI > Answer Insights. Feedback Display:
Example: 20 feedback entries with 16 positive and 4 negative displays as green with count of 16.
Detailed Feedback Analysis
- Click a query in Answer Insights to view the Answer Summary page
- See all answers users received for that query with associated feedback
- Click View Details for any answer to see user comments
Implementation Notes
When using SearchAINode:- Ensure
searchRequestIdis included in the channel response - Automatic when SearchAINode response is presented directly
- Must be explicitly included if response is saved to context and rendered with a custom template. Learn More.
Feedback vs. Feedback Surveys
These are separate mechanisms — the Search AI feedback mechanism is specifically designed for answer quality evaluation.
Quick Reference
Best Practices Checklist
Data Ingestion- Use digitally created, unencrypted documents
- Maintain consistent single-column layouts
- Add text descriptions for images and tables
- Use filters to ingest only relevant data
- Implement incremental updates
- Choose chunk size based on content type and use case
- Use 300-500 tokens for precise retrieval
- Use 1000+ tokens for complex narratives
- Select model based on budget vs. performance needs
- Ensure context window matches your chunk strategy
- Configure “Max tokens for Chunks” appropriately
- Monitor token usage and costs
- Test configurations to avoid context overflow