[{"data":1,"prerenderedAt":63},["Reactive",2],{"blog-post-markdown-dynamic-content-generation-automation-workflows-complete-guide":3},{"slug":4,"title":5,"description":6,"excerpt":7,"date":8,"category":9,"author":10,"keywords":11,"readingMinutes":12,"html":13,"headings":14,"tools":37,"related":38},"markdown-dynamic-content-generation-automation-workflows-complete-guide","Markdown Dynamic Content Generation and Automation Workflows: Complete Guide for Automated Documentation, Template Systems, and Content Pipeline Management","Master dynamic content generation in Markdown with comprehensive automation workflows, template engines, data-driven documentation systems, and intelligent content pipeline management for scalable technical writing.","Advanced Markdown dynamic content generation and automation workflows enable systematic creation of data-driven documentation, template-based content systems, and intelligent content pipelines…","2025-12-22","Tutorial","Matthew Rathbone","markdown automation, dynamic content generation markdown, markdown templates, automated documentation markdown, content pipeline markdown, markdown workflows, data driven documentation, template engines markdown",37,"\u003Cp>Advanced Markdown dynamic content generation and automation workflows enable systematic creation of data-driven documentation, template-based content systems, and intelligent content pipelines that automatically generate, update, and maintain documentation at scale. By implementing comprehensive automation strategies, template engines, and content generation systems, technical writers can build documentation ecosystems that adapt dynamically to changing data sources while maintaining consistency, accuracy, and efficiency across large content repositories.\u003C/p>\n\u003Ch2 id=\"why-master-dynamic-content-generation\">Why Master Dynamic Content Generation?\u003C/h2>\n\u003Cp>Professional content automation provides essential benefits for scalable documentation systems:\u003C/p>\n\u003Cul>\n\u003Cli>\u003Cstrong>Scalability\u003C/strong>: Generate thousands of documentation pages automatically from data sources\u003C/li>\n\u003Cli>\u003Cstrong>Consistency\u003C/strong>: Maintain uniform formatting and structure across all generated content\u003C/li>\n\u003Cli>\u003Cstrong>Real-time Updates\u003C/strong>: Automatically refresh documentation when underlying data changes\u003C/li>\n\u003Cli>\u003Cstrong>Efficiency\u003C/strong>: Reduce manual content creation time by 80-90% for repetitive documentation\u003C/li>\n\u003Cli>\u003Cstrong>Data Integration\u003C/strong>: Connect documentation directly to APIs, databases, and configuration systems\u003C/li>\n\u003Cli>\u003Cstrong>Quality Assurance\u003C/strong>: Implement automated validation and error checking in content pipelines\u003C/li>\n\u003C/ul>\n\u003Ch2 id=\"foundation-automation-principles\">Foundation Automation Principles\u003C/h2>\n\u003Ch3 id=\"template-driven-content-architecture\">Template-Driven Content Architecture\u003C/h3>\n\u003Cp>Understanding the core components of automated content generation:\u003C/p>\n\u003Cpre>\u003Ccode class=\"hljs language-markdown\">\u003Cspan class=\"hljs-section\"># Dynamic Content Generation Architecture\u003C/span>\n\n\u003Cspan class=\"hljs-section\">## Template Engine Foundation\u003C/span>\n\u003Cspan class=\"hljs-code\">```yaml\n# content-config.yml - Content generation configuration\nproject:\n  name: "API Documentation Generator"\n  version: "2.1.0"\n  base_url: "https://api.example.com"\n\ndata_sources:\n  - name: "api_endpoints"\n    type: "openapi"\n    source: "https://api.example.com/openapi.json"\n    refresh_interval: "hourly"\n    \n  - name: "user_metrics"\n    type: "database"\n    connection: "postgresql://metrics:5432/analytics"\n    query: "SELECT * FROM daily_metrics WHERE date >= NOW() - INTERVAL '30 days'"\n    refresh_interval: "daily"\n    \n  - name: "feature_flags"\n    type: "json"\n    source: "config/features.json"\n    refresh_interval: "on_change"\n\ntemplates:\n  - name: "api_endpoint_doc"\n    source: "templates/api-endpoint.md.j2"\n    output_pattern: "docs/api/endpoints/{{endpoint.path|slug}}.md"\n    data_source: "api_endpoints"\n    \n  - name: "metrics_report"\n    source: "templates/metrics-dashboard.md.j2"\n    output: "docs/analytics/daily-report.md"\n    data_source: "user_metrics"\n    \n  - name: "feature_documentation"\n    source: "templates/feature-guide.md.j2"\n    output_pattern: "docs/features/{{feature.name|slug}}.md"\n    data_source: "feature_flags"\n\nautomation:\n  triggers:\n    - type: "schedule"\n      cron: "0 */6 * * *"  # Every 6 hours\n      templates: ["api_endpoint_doc", "metrics_report"]\n      \n    - type: "webhook"\n      endpoint: "/regenerate"\n      secret: "${WEBHOOK_SECRET}"\n      templates: ["feature_documentation"]\n      \n    - type: "file_watch"\n      paths: ["config/**/*.json"]\n      templates: ["feature_documentation"]\n\nvalidation:\n  enabled: true\n  rules:\n    - check: "required_frontmatter"\n      fields: ["title", "description", "category"]\n    - check: "link_validation"\n      external_links: false\n    - check: "content_length"\n      min_words: 50\n```\u003C/span>\n\n\u003Cspan class=\"hljs-section\">## Basic Template Example\u003C/span>\n\n\u003Cspan class=\"hljs-code\">```jinja2\n<!-- templates/api-endpoint.md.j2 -->\n---\ntitle: "{{endpoint.summary}} - {{endpoint.method|upper}} {{endpoint.path}}"\ndescription: "{{endpoint.description|truncate(160)}}"\ncategory: "API Reference"\ntags: \n  - "{{endpoint.tags|join('", "')}}"\nmethod: "{{endpoint.method|upper}}"\npath: "{{endpoint.path}}"\n---\n\n# {{endpoint.summary}}\n\n{{endpoint.description}}\n\n## Request Details\n\n**Method:** `{{endpoint.method|upper}}`  \n**Path:** `{{endpoint.path}}`  \n**Content-Type:** `{{endpoint.consumes|default(['application/json'])|join(', ')}}`\n\n{% if endpoint.parameters %}\n## Parameters\n\n| Name | Type | Required | Description |\n|:-----|:-----|:---------|:------------|\n{% for param in endpoint.parameters %}\n| **{{param.name}}** | {{param.type}} | {{param.required|yesno}} | {{param.description|default('No description provided')}} |\n{% endfor %}\n{% endif %}\n\n{% if endpoint.requestBody %}\n## Request Body\n\n{{endpoint.requestBody.description}}\n\n### Example Request\n```\u003C/span>json\n{{endpoint.requestBody.examples.default|tojsonpretty}}\n\u003Cspan class=\"hljs-code\">```\n{% endif %}\n\n## Response\n\n{% for status_code, response in endpoint.responses.items() %}\n### {{status_code}} - {{response.description}}\n\n```\u003C/span>json\n{{response.examples.default|tojsonpretty}}\n\u003Cspan class=\"hljs-code\">```\n{% endfor %}\n\n## Code Examples\n\n### cURL\n```\u003C/span>bash\ncurl -X {{endpoint.method|upper}} \\\n  "{{project.base\u003Cspan class=\"hljs-emphasis\">_url}}{{endpoint.path}}" \\\n{% if endpoint.security %}\n  -H "Authorization: Bearer YOUR_\u003C/span>API\u003Cspan class=\"hljs-emphasis\">_KEY" \\\n{% endif %}\n  -H "Content-Type: application/json"\n{% if endpoint.requestBody %}\n  -d '{{endpoint.requestBody.examples.default|tojson}}'\n{% endif %}\n```\n\n### JavaScript\n```javascript\nconst response = await fetch('{{project.base_\u003C/span>url}}{{endpoint.path}}', {\n  method: '{{endpoint.method|upper}}',\n  headers: {\n\u003Cspan class=\"hljs-code\">    'Content-Type': 'application/json',\n{% if endpoint.security %}\n    'Authorization': 'Bearer YOUR_API_KEY',\n{% endif %}\n  },\n{% if endpoint.requestBody %}\n  body: JSON.stringify({{endpoint.requestBody.examples.default|tojsonpretty|indent(4)}})\n{% endif %}\n});\n\u003C/span>\nconst data = await response.json();\nconsole.log(data);\n\u003Cspan class=\"hljs-code\">```\n\n### Python\n```\u003C/span>python\nimport requests\n\nurl = "{{project.base\u003Cspan class=\"hljs-emphasis\">_url}}{{endpoint.path}}"\nheaders = {\n    "Content-Type": "application/json",\n{% if endpoint.security %}\n    "Authorization": "Bearer YOUR_\u003C/span>API\u003Cspan class=\"hljs-emphasis\">_KEY"\n{% endif %}\n}\n\n{% if endpoint.requestBody %}\ndata = {{endpoint.requestBody.examples.default|tojsonpretty|indent(4)}}\n\nresponse = requests.{{endpoint.method}}(url, headers=headers, json=data)\n{% else %}\nresponse = requests.{{endpoint.method}}(url, headers=headers)\n{% endif %}\nprint(response.json())\n```\n\n---\n*Generated automatically from OpenAPI specification on {{generation_\u003C/span>timestamp}}*\n\u003Cspan class=\"hljs-code\">```\n\u003C/span>\n\u003C/code>\u003C/pre>\u003Ch3 id=\"advanced-template-engine-implementation\">Advanced Template Engine Implementation\u003C/h3>\n\u003Cp>Building comprehensive template processing systems with multiple data sources:\u003C/p>\n\u003Cpre>\u003Ccode class=\"hljs language-markdown\">\u003Cspan class=\"hljs-section\"># Advanced Template Engine System\u003C/span>\n\n\u003Cspan class=\"hljs-section\">## Python-Based Generation Engine\u003C/span>\n\u003Cspan class=\"hljs-code\">```python\n#!/usr/bin/env python3\n# content-generator.py - Advanced dynamic content generation system\n\nimport os\nimport json\nimport yaml\nimport sqlite3\nimport asyncio\nimport aiohttp\nimport jinja2\nfrom jinja2 import Environment, FileSystemLoader, StrictUndefined\nfrom pathlib import Path\nfrom typing import Dict, Any, List, Optional, Union\nfrom datetime import datetime, timedelta\nimport hashlib\nimport logging\nfrom dataclasses import dataclass, field\nimport schedule\nimport time\nimport threading\nfrom watchdog.observers import Observer\nfrom watchdog.events import FileSystemEventHandler\nimport psycopg2\nfrom psycopg2.extras import RealDictCursor\nimport requests\nfrom urllib.parse import urljoin\n\nlogging.basicConfig(level=logging.INFO)\nlogger = logging.getLogger(__name__)\n\n@dataclass\nclass DataSource:\n    """Configuration for data source"""\n    name: str\n    type: str\n    source: str\n    refresh_interval: str\n    connection_params: Dict[str, Any] = field(default_factory=dict)\n    query: Optional[str] = None\n    transform_function: Optional[str] = None\n    cache_duration: int = 3600  # seconds\n    \n@dataclass\nclass Template:\n    """Template configuration"""\n    name: str\n    source: str\n    output_pattern: Optional[str] = None\n    output: Optional[str] = None\n    data_source: str = ""\n    context_processors: List[str] = field(default_factory=list)\n    conditions: Dict[str, Any] = field(default_factory=dict)\n\n@dataclass\nclass GenerationResult:\n    """Result of content generation"""\n    template_name: str\n    output_path: str\n    success: bool\n    error_message: Optional[str] = None\n    generated_at: datetime = field(default_factory=datetime.now)\n    data_hash: Optional[str] = None\n\nclass ContentGenerationEngine:\n    """Advanced content generation engine with multiple data sources"""\n    \n    def __init__(self, config_path: str):\n        self.config_path = Path(config_path)\n        self.config = self._load_config()\n        self.data_cache = {}\n        self.cache_timestamps = {}\n        \n        # Setup Jinja2 environment\n        self.jinja_env = Environment(\n            loader=FileSystemLoader(['templates', '.']),\n            undefined=StrictUndefined,\n            trim_blocks=True,\n            lstrip_blocks=True\n        )\n        \n        # Add custom filters\n        self._setup_custom_filters()\n        \n        # Initialize data sources\n        self.data_sources = {\n            source_config['name']: DataSource(**source_config)\n            for source_config in self.config.get('data_sources', [])\n        }\n        \n        # Initialize templates\n        self.templates = {\n            template_config['name']: Template(**template_config)\n            for template_config in self.config.get('templates', [])\n        }\n        \n        # Setup automation\n        self._setup_automation()\n        \n    def _load_config(self) -> Dict[str, Any]:\n        """Load configuration from YAML file"""\n        try:\n            with open(self.config_path, 'r') as f:\n                return yaml.safe_load(f)\n        except Exception as e:\n            logger.error(f"Error loading config: {e}")\n            raise\n    \n    def _setup_custom_filters(self):\n        """Setup custom Jinja2 filters"""\n        \n        def slug_filter(text: str) -> str:\n            """Convert text to URL-friendly slug"""\n            import re\n            text = re.sub(r'[^\\w\\s-]', '', str(text)).strip().lower()\n            return re.sub(r'[\\s_-]+', '-', text)\n        \n        def yesno_filter(value: bool) -> str:\n            """Convert boolean to Yes/No"""\n            return "Yes" if value else "No"\n        \n        def tojsonpretty_filter(obj: Any) -> str:\n            """Convert object to pretty JSON"""\n            return json.dumps(obj, indent=2, ensure_ascii=False)\n        \n        def truncate_words_filter(text: str, length: int = 50) -> str:\n            """Truncate text to specified word count"""\n            words = str(text).split()\n            if len(words) <= length:\n                return text\n            return ' '.join(words[:length]) + '...'\n        \n        def relative_time_filter(dt: datetime) -> str:\n            """Convert datetime to relative time string"""\n            now = datetime.now()\n            diff = now - dt\n            \n            if diff.days > 0:\n                return f"{diff.days} day{'s' if diff.days > 1 else ''} ago"\n            elif diff.seconds > 3600:\n                hours = diff.seconds // 3600\n                return f"{hours} hour{'s' if hours > 1 else ''} ago"\n            elif diff.seconds > 60:\n                minutes = diff.seconds // 60\n                return f"{minutes} minute{'s' if minutes > 1 else ''} ago"\n            else:\n                return "just now"\n        \n        def format_number_filter(value: Union[int, float], format_type: str = 'default') -> str:\n            """Format numbers with various styles"""\n            if format_type == 'currency':\n                return f"${value:,.2f}"\n            elif format_type == 'percentage':\n                return f"{value:.1f}%"\n            elif format_type == 'abbreviated':\n                if value >= 1_000_000:\n                    return f"{value/1_000_000:.1f}M"\n                elif value >= 1_000:\n                    return f"{value/1_000:.1f}K"\n                else:\n                    return str(value)\n            else:\n                return f"{value:,}"\n        \n        # Register filters\n        self.jinja_env.filters['slug'] = slug_filter\n        self.jinja_env.filters['yesno'] = yesno_filter\n        self.jinja_env.filters['tojsonpretty'] = tojsonpretty_filter\n        self.jinja_env.filters['truncate_words'] = truncate_words_filter\n        self.jinja_env.filters['relative_time'] = relative_time_filter\n        self.jinja_env.filters['format_number'] = format_number_filter\n        \n        # Add global functions\n        self.jinja_env.globals['now'] = datetime.now\n        self.jinja_env.globals['generation_timestamp'] = datetime.now().isoformat()\n    \n    async def fetch_data_source(self, source: DataSource) -> Dict[str, Any]:\n        """Fetch data from configured source"""\n        cache_key = source.name\n        now = time.time()\n        \n        # Check cache validity\n        if (cache_key in self.cache_timestamps and \n            now - self.cache_timestamps[cache_key] < source.cache_duration):\n            logger.info(f"Using cached data for {source.name}")\n            return self.data_cache[cache_key]\n        \n        logger.info(f"Fetching fresh data for {source.name}")\n        \n        try:\n            if source.type == 'openapi':\n                data = await self._fetch_openapi_data(source)\n            elif source.type == 'database':\n                data = await self._fetch_database_data(source)\n            elif source.type == 'json':\n                data = await self._fetch_json_data(source)\n            elif source.type == 'api':\n                data = await self._fetch_api_data(source)\n            elif source.type == 'csv':\n                data = await self._fetch_csv_data(source)\n            else:\n                raise ValueError(f"Unknown data source type: {source.type}")\n            \n            # Apply transformation if specified\n            if source.transform_function:\n                data = await self._apply_transformation(data, source.transform_function)\n            \n            # Cache the data\n            self.data_cache[cache_key] = data\n            self.cache_timestamps[cache_key] = now\n            \n            return data\n            \n        except Exception as e:\n            logger.error(f"Error fetching data from {source.name}: {e}")\n            # Return cached data if available, otherwise raise\n            if cache_key in self.data_cache:\n                logger.warning(f"Using stale cached data for {source.name}")\n                return self.data_cache[cache_key]\n            raise\n    \n    async def _fetch_openapi_data(self, source: DataSource) -> Dict[str, Any]:\n        """Fetch and process OpenAPI specification"""\n        async with aiohttp.ClientSession() as session:\n            async with session.get(source.source) as response:\n                response.raise_for_status()\n                spec = await response.json()\n        \n        # Process OpenAPI spec into template-friendly format\n        endpoints = []\n        \n        for path, path_methods in spec.get('paths', {}).items():\n            for method, operation in path_methods.items():\n                if method.lower() in ['get', 'post', 'put', 'patch', 'delete']:\n                    endpoint = {\n                        'path': path,\n                        'method': method.lower(),\n                        'summary': operation.get('summary', ''),\n                        'description': operation.get('description', ''),\n                        'tags': operation.get('tags', []),\n                        'parameters': operation.get('parameters', []),\n                        'requestBody': operation.get('requestBody'),\n                        'responses': operation.get('responses', {}),\n                        'security': operation.get('security', [])\n                    }\n                    \n                    # Process examples\n                    if endpoint['requestBody']:\n                        content = endpoint['requestBody'].get('content', {})\n                        for media_type, schema in content.items():\n                            if 'examples' not in schema and 'example' in schema:\n                                schema['examples'] = {'default': schema['example']}\n                    \n                    endpoints.append(endpoint)\n        \n        return {\n            'spec': spec,\n            'endpoints': endpoints,\n            'info': spec.get('info', {}),\n            'servers': spec.get('servers', [])\n        }\n    \n    async def _fetch_database_data(self, source: DataSource) -> Dict[str, Any]:\n        """Fetch data from database"""\n        connection_string = source.source\n        \n        if connection_string.startswith('postgresql://'):\n            import psycopg2.pool\n            \n            # Create connection pool for better performance\n            pool = psycopg2.pool.SimpleConnectionPool(\n                1, 5, connection_string,\n                cursor_factory=RealDictCursor\n            )\n            \n            try:\n                conn = pool.getconn()\n                with conn.cursor() as cursor:\n                    cursor.execute(source.query)\n                    results = cursor.fetchall()\n                    \n                    # Convert to list of dictionaries\n                    data = [dict(row) for row in results]\n                    \n                return {'rows': data, 'count': len(data)}\n                \n            finally:\n                pool.putconn(conn)\n                pool.closeall()\n        \n        elif connection_string.startswith('sqlite://'):\n            db_path = connection_string.replace('sqlite://', '')\n            \n            conn = sqlite3.connect(db_path)\n            conn.row_factory = sqlite3.Row\n            \n            try:\n                cursor = conn.cursor()\n                cursor.execute(source.query)\n                results = cursor.fetchall()\n                \n                data = [dict(row) for row in results]\n                return {'rows': data, 'count': len(data)}\n                \n            finally:\n                conn.close()\n        \n        else:\n            raise ValueError(f"Unsupported database type in: {connection_string}")\n    \n    async def _fetch_json_data(self, source: DataSource) -> Dict[str, Any]:\n        """Fetch JSON data from file or URL"""\n        if source.source.startswith('http'):\n            async with aiohttp.ClientSession() as session:\n                async with session.get(source.source) as response:\n                    response.raise_for_status()\n                    return await response.json()\n        else:\n            with open(source.source, 'r') as f:\n                return json.load(f)\n    \n    async def _fetch_api_data(self, source: DataSource) -> Dict[str, Any]:\n        """Fetch data from REST API"""\n        headers = source.connection_params.get('headers', {})\n        params = source.connection_params.get('params', {})\n        \n        async with aiohttp.ClientSession() as session:\n            async with session.get(\n                source.source, \n                headers=headers, \n                params=params\n            ) as response:\n                response.raise_for_status()\n                return await response.json()\n    \n    async def _fetch_csv_data(self, source: DataSource) -> Dict[str, Any]:\n        """Fetch and process CSV data"""\n        import pandas as pd\n        \n        if source.source.startswith('http'):\n            df = pd.read_csv(source.source)\n        else:\n            df = pd.read_csv(source.source)\n        \n        # Convert to records format\n        records = df.to_dict('records')\n        \n        return {\n            'records': records,\n            'columns': df.columns.tolist(),\n            'count': len(records),\n            'summary': {\n                'total_rows': len(records),\n                'columns': len(df.columns),\n                'numeric_columns': len(df.select_dtypes(include=['number']).columns),\n                'text_columns': len(df.select_dtypes(include=['object']).columns)\n            }\n        }\n    \n    async def _apply_transformation(self, data: Dict[str, Any], transform_function: str) -> Dict[str, Any]:\n        """Apply custom transformation function to data"""\n        # This would load and execute custom transformation functions\n        # For security, this should be sandboxed in production\n        \n        # Example: Simple transformation functions\n        if transform_function == 'sort_by_name':\n            if 'endpoints' in data:\n                data['endpoints'] = sorted(data['endpoints'], key=lambda x: x.get('summary', ''))\n        elif transform_function == 'group_by_tag':\n            if 'endpoints' in data:\n                grouped = {}\n                for endpoint in data['endpoints']:\n                    for tag in endpoint.get('tags', ['default']):\n                        if tag not in grouped:\n                            grouped[tag] = []\n                        grouped[tag].append(endpoint)\n                data['grouped_endpoints'] = grouped\n        \n        return data\n    \n    async def generate_content(self, template_name: str, force: bool = False) -> List[GenerationResult]:\n        """Generate content for specified template"""\n        if template_name not in self.templates:\n            raise ValueError(f"Template {template_name} not found")\n        \n        template = self.templates[template_name]\n        results = []\n        \n        try:\n            # Load template\n            jinja_template = self.jinja_env.get_template(template.source)\n            \n            # Fetch data\n            data = {}\n            if template.data_source:\n                source = self.data_sources[template.data_source]\n                data = await self.fetch_data_source(source)\n            \n            # Apply context processors\n            context = await self._build_context(data, template)\n            \n            # Check conditions\n            if not self._check_conditions(context, template):\n                logger.info(f"Conditions not met for template {template_name}")\n                return results\n            \n            # Generate content\n            if template.output_pattern:\n                # Generate multiple files based on pattern\n                results.extend(await self._generate_multiple_files(jinja_template, context, template))\n            else:\n                # Generate single file\n                result = await self._generate_single_file(jinja_template, context, template)\n                results.append(result)\n            \n        except Exception as e:\n            logger.error(f"Error generating content for {template_name}: {e}")\n            results.append(GenerationResult(\n                template_name=template_name,\n                output_path="",\n                success=False,\n                error_message=str(e)\n            ))\n        \n        return results\n    \n    async def _build_context(self, data: Dict[str, Any], template: Template) -> Dict[str, Any]:\n        """Build template context with all required data"""\n        context = {\n            'data': data,\n            'config': self.config,\n            'generation_time': datetime.now(),\n            'template_name': template.name\n        }\n        \n        # Apply context processors\n        for processor_name in template.context_processors:\n            context = await self._apply_context_processor(context, processor_name)\n        \n        return context\n    \n    async def _apply_context_processor(self, context: Dict[str, Any], processor_name: str) -> Dict[str, Any]:\n        """Apply context processor to enhance template data"""\n        # Example context processors\n        if processor_name == 'add_navigation':\n            context['navigation'] = self._generate_navigation_data(context)\n        elif processor_name == 'add_metadata':\n            context['metadata'] = self._generate_metadata(context)\n        elif processor_name == 'add_related_content':\n            context['related'] = await self._find_related_content(context)\n        \n        return context\n    \n    def _generate_navigation_data(self, context: Dict[str, Any]) -> Dict[str, Any]:\n        """Generate navigation data for templates"""\n        return {\n            'sections': [\n                {'name': 'API Reference', 'url': '/api/'},\n                {'name': 'Guides', 'url': '/guides/'},\n                {'name': 'Examples', 'url': '/examples/'}\n            ],\n            'breadcrumbs': self._generate_breadcrumbs(context)\n        }\n    \n    def _generate_breadcrumbs(self, context: Dict[str, Any]) -> List[Dict[str, str]]:\n        """Generate breadcrumb navigation"""\n        # Implementation would depend on the specific content structure\n        return [\n            {'name': 'Home', 'url': '/'},\n            {'name': 'Documentation', 'url': '/docs/'}\n        ]\n    \n    def _generate_metadata(self, context: Dict[str, Any]) -> Dict[str, Any]:\n        """Generate metadata for content"""\n        return {\n            'generated_at': datetime.now().isoformat(),\n            'generator_version': '2.1.0',\n            'data_sources': list(self.data_sources.keys()),\n            'last_updated': context.get('generation_time', datetime.now()).isoformat()\n        }\n    \n    async def _find_related_content(self, context: Dict[str, Any]) -> List[Dict[str, Any]]:\n        """Find related content based on context"""\n        # This would implement content similarity algorithms\n        return []\n    \n    def _check_conditions(self, context: Dict[str, Any], template: Template) -> bool:\n        """Check if template conditions are met"""\n        if not template.conditions:\n            return True\n        \n        for condition_key, condition_value in template.conditions.items():\n            if condition_key == 'min_data_count':\n                data_count = len(context.get('data', {}).get('rows', []))\n                if data_count < condition_value:\n                    return False\n            elif condition_key == 'required_fields':\n                data = context.get('data', {})\n                for field in condition_value:\n                    if field not in data:\n                        return False\n        \n        return True\n    \n    async def _generate_single_file(self, jinja_template, context: Dict[str, Any], template: Template) -> GenerationResult:\n        """Generate single output file"""\n        try:\n            content = jinja_template.render(**context)\n            \n            # Create output directory if needed\n            output_path = Path(template.output)\n            output_path.parent.mkdir(parents=True, exist_ok=True)\n            \n            # Write file\n            with open(output_path, 'w', encoding='utf-8') as f:\n                f.write(content)\n            \n            # Calculate content hash for change detection\n            content_hash = hashlib.sha256(content.encode()).hexdigest()\n            \n            logger.info(f"Generated {output_path}")\n            \n            return GenerationResult(\n                template_name=template.name,\n                output_path=str(output_path),\n                success=True,\n                data_hash=content_hash\n            )\n            \n        except Exception as e:\n            return GenerationResult(\n                template_name=template.name,\n                output_path=template.output or "",\n                success=False,\n                error_message=str(e)\n            )\n    \n    async def _generate_multiple_files(self, jinja_template, context: Dict[str, Any], template: Template) -> List[GenerationResult]:\n        """Generate multiple files based on output pattern"""\n        results = []\n        \n        try:\n            # Extract iterable data\n            data = context.get('data', {})\n            \n            if 'endpoints' in data:\n                items = data['endpoints']\n                item_key = 'endpoint'\n            elif 'records' in data:\n                items = data['records']\n                item_key = 'record'\n            elif 'rows' in data:\n                items = data['rows']\n                item_key = 'row'\n            else:\n                items = [data]  # Single item\n                item_key = 'item'\n            \n            # Generate file for each item\n            for item in items:\n                item_context = context.copy()\n                item_context[item_key] = item\n                \n                # Render output path pattern\n                output_pattern_template = self.jinja_env.from_string(template.output_pattern)\n                output_path = output_pattern_template.render(**item_context)\n                \n                # Generate content\n                content = jinja_template.render(**item_context)\n                \n                # Create output directory\n                output_path_obj = Path(output_path)\n                output_path_obj.parent.mkdir(parents=True, exist_ok=True)\n                \n                # Write file\n                with open(output_path_obj, 'w', encoding='utf-8') as f:\n                    f.write(content)\n                \n                content_hash = hashlib.sha256(content.encode()).hexdigest()\n                \n                results.append(GenerationResult(\n                    template_name=template.name,\n                    output_path=str(output_path_obj),\n                    success=True,\n                    data_hash=content_hash\n                ))\n                \n                logger.info(f"Generated {output_path_obj}")\n            \n        except Exception as e:\n            logger.error(f"Error in multiple file generation: {e}")\n            results.append(GenerationResult(\n                template_name=template.name,\n                output_path="",\n                success=False,\n                error_message=str(e)\n            ))\n        \n        return results\n    \n    def _setup_automation(self):\n        """Setup automation triggers"""\n        automation_config = self.config.get('automation', {})\n        triggers = automation_config.get('triggers', [])\n        \n        for trigger in triggers:\n            if trigger['type'] == 'schedule':\n                self._setup_scheduled_trigger(trigger)\n            elif trigger['type'] == 'file_watch':\n                self._setup_file_watch_trigger(trigger)\n            elif trigger['type'] == 'webhook':\n                self._setup_webhook_trigger(trigger)\n    \n    def _setup_scheduled_trigger(self, trigger_config: Dict[str, Any]):\n        """Setup scheduled content generation"""\n        cron_expression = trigger_config['cron']\n        templates = trigger_config['templates']\n        \n        def job():\n            asyncio.run(self._run_scheduled_job(templates))\n        \n        # Simple cron parsing (in production, use croniter or similar)\n        if cron_expression == "0 */6 * * *":  # Every 6 hours\n            schedule.every(6).hours.do(job)\n        elif cron_expression == "0 0 * * *":  # Daily at midnight\n            schedule.every().day.at("00:00").do(job)\n        \n        # Start scheduler in separate thread\n        def run_scheduler():\n            while True:\n                schedule.run_pending()\n                time.sleep(60)\n        \n        scheduler_thread = threading.Thread(target=run_scheduler, daemon=True)\n        scheduler_thread.start()\n    \n    async def _run_scheduled_job(self, templates: List[str]):\n        """Run scheduled content generation job"""\n        logger.info(f"Running scheduled job for templates: {templates}")\n        \n        for template_name in templates:\n            try:\n                results = await self.generate_content(template_name)\n                for result in results:\n                    if result.success:\n                        logger.info(f"Successfully generated {result.output_path}")\n                    else:\n                        logger.error(f"Failed to generate content: {result.error_message}")\n            except Exception as e:\n                logger.error(f"Error in scheduled job for {template_name}: {e}")\n    \n    def _setup_file_watch_trigger(self, trigger_config: Dict[str, Any]):\n        """Setup file system watcher for automatic regeneration"""\n        paths = trigger_config['paths']\n        templates = trigger_config['templates']\n        \n        class FileChangeHandler(FileSystemEventHandler):\n            def __init__(self, generator):\n                self.generator = generator\n                self.templates = templates\n            \n            def on_modified(self, event):\n                if not event.is_directory:\n                    logger.info(f"File changed: {event.src_path}")\n                    asyncio.run(self.generator._run_scheduled_job(self.templates))\n        \n        event_handler = FileChangeHandler(self)\n        observer = Observer()\n        \n        for path in paths:\n            observer.schedule(event_handler, path, recursive=True)\n        \n        observer.start()\n        logger.info(f"Started file watcher for paths: {paths}")\n    \n    def _setup_webhook_trigger(self, trigger_config: Dict[str, Any]):\n        """Setup webhook endpoint for content regeneration"""\n        # This would typically integrate with a web framework like Flask/FastAPI\n        # For simplicity, this is just a placeholder\n        logger.info(f"Webhook trigger configured for endpoint: {trigger_config['endpoint']}")\n    \n    async def generate_all_content(self, force: bool = False) -> Dict[str, List[GenerationResult]]:\n        """Generate content for all configured templates"""\n        all_results = {}\n        \n        for template_name in self.templates:\n            try:\n                results = await self.generate_content(template_name, force=force)\n                all_results[template_name] = results\n            except Exception as e:\n                logger.error(f"Error generating {template_name}: {e}")\n                all_results[template_name] = [GenerationResult(\n                    template_name=template_name,\n                    output_path="",\n                    success=False,\n                    error_message=str(e)\n                )]\n        \n        return all_results\n    \n    def get_generation_stats(self) -> Dict[str, Any]:\n        """Get statistics about content generation"""\n        total_templates = len(self.templates)\n        active_data_sources = len([\n            source for source in self.data_sources.values()\n            if source.name in self.cache_timestamps\n        ])\n        \n        cache_info = {}\n        for source_name, timestamp in self.cache_timestamps.items():\n            age_seconds = time.time() - timestamp\n            cache_info[source_name] = {\n                'age_seconds': age_seconds,\n                'age_minutes': age_seconds / 60,\n                'is_fresh': age_seconds < self.data_sources[source_name].cache_duration\n            }\n        \n        return {\n            'total_templates': total_templates,\n            'active_data_sources': active_data_sources,\n            'cache_info': cache_info,\n            'last_generation': datetime.now().isoformat()\n        }\n\n# CLI Interface\nasync def main():\n    import argparse\n    \n    parser = argparse.ArgumentParser(description='Dynamic Content Generator')\n    parser.add_argument('--config', required=True, help='Configuration file path')\n    parser.add_argument('--template', help='Generate specific template')\n    parser.add_argument('--force', action='store_true', help='Force regeneration ignoring cache')\n    parser.add_argument('--stats', action='store_true', help='Show generation statistics')\n    \n    args = parser.parse_args()\n    \n    generator = ContentGenerationEngine(args.config)\n    \n    if args.stats:\n        stats = generator.get_generation_stats()\n        print(json.dumps(stats, indent=2))\n        return\n    \n    if args.template:\n        results = await generator.generate_content(args.template, force=args.force)\n    else:\n        all_results = await generator.generate_all_content(force=args.force)\n        results = []\n        for template_results in all_results.values():\n            results.extend(template_results)\n    \n    # Print summary\n    successful = len([r for r in results if r.success])\n    failed = len([r for r in results if not r.success])\n    \n    print(f"Generation complete: {successful} successful, {failed} failed")\n    \n    for result in results:\n        if result.success:\n            print(f"✅ {result.output_path}")\n        else:\n            print(f"❌ {result.template_name}: {result.error_message}")\n\nif __name__ == "__main__":\n    asyncio.run(main())\n```\u003C/span>\n\u003C/code>\u003C/pre>\u003Ch2 id=\"integration-with-cicd-systems\">Integration with CI/CD Systems\u003C/h2>\n\u003Ch3 id=\"github-actions-workflow-integration\">GitHub Actions Workflow Integration\u003C/h3>\n\u003Cp>Implementing automated content generation in continuous integration pipelines:\u003C/p>\n\u003Cpre>\u003Ccode class=\"hljs language-markdown\">\u003Cspan class=\"hljs-section\"># CI/CD Integration for Dynamic Content\u003C/span>\n\n\u003Cspan class=\"hljs-section\">## GitHub Actions Workflow\u003C/span>\n\n\u003Cspan class=\"hljs-code\">```yaml\n# .github/workflows/content-generation.yml\nname: Dynamic Content Generation\n\non:\n  push:\n    branches: [ main, develop ]\n    paths:\n      - 'data/**'\n      - 'templates/**'\n      - 'content-config.yml'\n  schedule:\n    - cron: '0 6 * * *'  # Daily at 6 AM\n  workflow_dispatch:\n    inputs:\n      force_regeneration:\n        description: 'Force full regeneration'\n        required: false\n        default: 'false'\n      specific_template:\n        description: 'Generate specific template only'\n        required: false\n\njobs:\n  generate-content:\n    runs-on: ubuntu-latest\n    \n    steps:\n    - uses: actions/checkout@v4\n      with:\n        token: ${{ secrets.GITHUB_TOKEN }}\n        fetch-depth: 0\n    \n    - name: Set up Python\n      uses: actions/setup-python@v4\n      with:\n        python-version: '3.11'\n    \n    - name: Install dependencies\n      run: |\n        pip install -r requirements.txt\n        pip install jinja2 aiohttp psycopg2-binary pandas pyyaml watchdog schedule\n    \n    - name: Configure data source credentials\n      run: |\n        echo "DATABASE_URL=${{ secrets.DATABASE_URL }}" >> $GITHUB_ENV\n        echo "API_KEY=${{ secrets.API_KEY }}" >> $GITHUB_ENV\n    \n    - name: Generate content\n      run: |\n        if [ "${{ github.event.inputs.specific_template }}" != "" ]; then\n          python content-generator.py \\\n            --config content-config.yml \\\n            --template "${{ github.event.inputs.specific_template }}" \\\n            ${{ github.event.inputs.force_regeneration == 'true' && '--force' || '' }}\n        else\n          python content-generator.py \\\n            --config content-config.yml \\\n            ${{ github.event.inputs.force_regeneration == 'true' && '--force' || '' }}\n        fi\n    \n    - name: Validate generated content\n      run: |\n        python scripts/validate-generated-content.py \\\n          --content-dir docs \\\n          --check-links \\\n          --check-frontmatter \\\n          --output validation-report.json\n    \n    - name: Build site for testing\n      run: |\n        bundle install\n        bundle exec jekyll build\n    \n    - name: Test generated content\n      run: |\n        python scripts/test-content-quality.py \\\n          --site-dir _site \\\n          --check-performance \\\n          --check-accessibility \\\n          --output test-results.json\n    \n    - name: Commit generated content\n      if: success()\n      run: |\n        git config --global user.name 'Content Generator Bot'\n        git config --global user.email 'bot@example.com'\n        \n        git add docs/\n        \n        if ! git diff --staged --quiet; then\n          git commit -m "🤖 Automated content generation\n          \n          Generated content for:\n          - API documentation\n          - Metrics reports\n          - Feature guides\n          \n          Triggered by: ${{ github.event_name }}\n          ${{ github.event.inputs.specific_template && format('Template: {0}', github.event.inputs.specific_template) || '' }}\n          \n          🤖 Generated with Content Pipeline\n          "\n          \n          git push\n        else\n          echo "No changes to commit"\n        fi\n    \n    - name: Upload generation artifacts\n      if: always()\n      uses: actions/upload-artifact@v3\n      with:\n        name: content-generation-report\n        path: |\n          validation-report.json\n          test-results.json\n          generation-stats.json\n\n  deploy-content:\n    runs-on: ubuntu-latest\n    needs: generate-content\n    if: github.ref == 'refs/heads/main'\n    \n    steps:\n    - uses: actions/checkout@v4\n      with:\n        ref: main  # Get the latest content\n    \n    - name: Deploy to production\n      run: |\n        # Deploy generated content to production environment\n        echo "Deploying generated content to production"\n        \n        # Example: Deploy to GitHub Pages\n        npm install -g @11ty/eleventy\n        eleventy\n    \n    - name: Notify stakeholders\n      if: success()\n      uses: 8398a7/action-slack@v3\n      with:\n        status: success\n        text: "📚 Documentation updated successfully with latest content generation"\n      env:\n        SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}\n    \n    - name: Update content metrics\n      run: |\n        python scripts/update-content-metrics.py \\\n          --generated-files docs/ \\\n          --analytics-endpoint "${{ secrets.ANALYTICS_ENDPOINT }}" \\\n          --api-key "${{ secrets.ANALYTICS_API_KEY }}"\n\n  content-quality-check:\n    runs-on: ubuntu-latest\n    needs: generate-content\n    \n    steps:\n    - uses: actions/checkout@v4\n    \n    - name: Quality assurance checks\n      run: |\n        # Check for content quality issues\n        python scripts/content-qa.py \\\n          --content-dir docs \\\n          --check-duplicates \\\n          --check-broken-references \\\n          --check-consistency \\\n          --output qa-report.html\n    \n    - name: Performance benchmark\n      run: |\n        # Benchmark content generation performance\n        python scripts/benchmark-generation.py \\\n          --config content-config.yml \\\n          --iterations 5 \\\n          --output benchmark-results.json\n    \n    - name: Update documentation metrics\n      run: |\n        # Update internal metrics about documentation\n        python scripts/update-docs-metrics.py \\\n          --benchmark-data benchmark-results.json \\\n          --qa-data qa-report.html \\\n          --target-endpoint "${{ secrets.METRICS_ENDPOINT }}"\n```\u003C/span>\n\n\n\u003Cspan class=\"hljs-section\">## Advanced Content Validation System\u003C/span>\n\u003Cspan class=\"hljs-code\">```python\n#!/usr/bin/env python3\n# validate-generated-content.py - Content validation for generated files\n\nimport os\nimport json\nimport yaml\nimport requests\nfrom pathlib import Path\nfrom typing import Dict, List, Any, Optional\nfrom dataclasses import dataclass\nimport frontmatter\nfrom urllib.parse import urljoin, urlparse\nimport re\nfrom datetime import datetime\nimport concurrent.futures\nimport threading\nimport time\n\n@dataclass\nclass ValidationResult:\n    """Result of content validation"""\n    file_path: str\n    validation_type: str\n    status: str  # 'passed', 'failed', 'warning'\n    message: str\n    details: Optional[Dict[str, Any]] = None\n\nclass ContentValidator:\n    """Comprehensive validation for generated content"""\n    \n    def __init__(self, content_dir: str, base_url: str = ""):\n        self.content_dir = Path(content_dir)\n        self.base_url = base_url\n        self.results = []\n        self.link_cache = {}\n        self.validation_stats = {\n            'total_files': 0,\n            'passed': 0,\n            'failed': 0,\n            'warnings': 0\n        }\n    \n    def validate_all_content(self, \n                           check_links: bool = True,\n                           check_frontmatter: bool = True,\n                           check_content_quality: bool = True,\n                           check_consistency: bool = True) -> List[ValidationResult]:\n        """Run all validation checks on generated content"""\n        \n        markdown_files = list(self.content_dir.glob('**/*.md'))\n        self.validation_stats['total_files'] = len(markdown_files)\n        \n        print(f"Validating {len(markdown_files)} files...")\n        \n        # Validate each file\n        for file_path in markdown_files:\n            try:\n                with open(file_path, 'r', encoding='utf-8') as f:\n                    content = f.read()\n                \n                # Parse frontmatter and content\n                post = frontmatter.loads(content)\n                \n                if check_frontmatter:\n                    self._validate_frontmatter(file_path, post)\n                \n                if check_content_quality:\n                    self._validate_content_quality(file_path, post)\n                \n                if check_consistency:\n                    self._validate_consistency(file_path, post)\n                \n                if check_links:\n                    self._validate_links(file_path, post)\n                    \n            except Exception as e:\n                self.results.append(ValidationResult(\n                    file_path=str(file_path),\n                    validation_type="file_processing",\n                    status="failed",\n                    message=f"Error processing file: {str(e)}"\n                ))\n        \n        # Additional cross-file validations\n        if check_consistency:\n            self._validate_cross_file_consistency(markdown_files)\n        \n        # Update statistics\n        self._calculate_stats()\n        \n        return self.results\n    \n    def _validate_frontmatter(self, file_path: Path, post) -> None:\n        """Validate frontmatter completeness and format"""\n        metadata = post.metadata\n        \n        # Required fields\n        required_fields = ['title', 'description', 'category', 'date']\n        missing_fields = [field for field in required_fields if field not in metadata]\n        \n        if missing_fields:\n            self.results.append(ValidationResult(\n                file_path=str(file_path),\n                validation_type="frontmatter",\n                status="failed",\n                message=f"Missing required frontmatter fields: {', '.join(missing_fields)}"\n            ))\n        \n        # Validate field formats\n        if 'date' in metadata:\n            try:\n                if isinstance(metadata['date'], str):\n                    datetime.strptime(metadata['date'], '%Y-%m-%d')\n            except ValueError:\n                self.results.append(ValidationResult(\n                    file_path=str(file_path),\n                    validation_type="frontmatter",\n                    status="failed",\n                    message="Invalid date format in frontmatter (expected YYYY-MM-DD)"\n                ))\n        \n        # Title length check\n        if 'title' in metadata and len(metadata['title']) > 200:\n            self.results.append(ValidationResult(\n                file_path=str(file_path),\n                validation_type="frontmatter",\n                status="warning",\n                message="Title is very long (over 200 characters)"\n            ))\n        \n        # Description length check\n        if 'description' in metadata:\n            desc_length = len(metadata['description'])\n            if desc_length < 50:\n                self.results.append(ValidationResult(\n                    file_path=str(file_path),\n                    validation_type="frontmatter",\n                    status="warning",\n                    message="Description is too short (under 50 characters)"\n                ))\n            elif desc_length > 300:\n                self.results.append(ValidationResult(\n                    file_path=str(file_path),\n                    validation_type="frontmatter",\n                    status="warning",\n                    message="Description is too long (over 300 characters)"\n                ))\n    \n    def _validate_content_quality(self, file_path: Path, post) -> None:\n        """Validate content quality metrics"""\n        content = post.content\n        \n        # Word count check\n        word_count = len(content.split())\n        if word_count < 100:\n            self.results.append(ValidationResult(\n                file_path=str(file_path),\n                validation_type="content_quality",\n                status="warning",\n                message=f"Content is very short ({word_count} words)"\n            ))\n        \n        # Check for placeholder content\n        placeholder_patterns = [\n            r'\\[placeholder\\]',\n            r'\\[todo\\]',\n            r'\\[tbd\\]',\n            r'lorem ipsum',\n            r'placeholder text'\n        ]\n        \n        for pattern in placeholder_patterns:\n            if re.search(pattern, content, re.IGNORECASE):\n                self.results.append(ValidationResult(\n                    file_path=str(file_path),\n                    validation_type="content_quality",\n                    status="failed",\n                    message=f"Placeholder content detected: {pattern}"\n                ))\n        \n        # Check for proper heading structure\n        headings = re.findall(r'^(#+)\\s+(.+)$', content, re.MULTILINE)\n        if headings:\n            # Check if starts with h1\n            first_heading_level = len(headings[0][0])\n            if first_heading_level != 1:\n                self.results.append(ValidationResult(\n                    file_path=str(file_path),\n                    validation_type="content_quality",\n                    status="warning",\n                    message="Document should start with h1 heading"\n                ))\n            \n            # Check heading hierarchy\n            prev_level = 0\n            for i, (hashes, text) in enumerate(headings):\n                level = len(hashes)\n                if level > prev_level + 1 and prev_level > 0:\n                    self.results.append(ValidationResult(\n                        file_path=str(file_path),\n                        validation_type="content_quality",\n                        status="warning",\n                        message=f"Heading hierarchy skip detected at line with '{text}'"\n                    ))\n                prev_level = level\n        \n        # Check for code block formatting\n        code_blocks = re.findall(r'```\u003C/span>(\\w+)?\\n(.\u003Cspan class=\"hljs-emphasis\">*?)\\n```', content, re.DOTALL)\n        for lang, code in code_blocks:\n            if not lang:\n                self.results.append(ValidationResult(\n                    file_path=str(file_path),\n                    validation_type="content_quality",\n                    status="warning",\n                    message="Code block without language specification"\n                ))\n            \n            # Check for unterminated code blocks\n            if '```' in code:\n                self.results.append(ValidationResult(\n                    file_path=str(file_path),\n                    validation_type="content_quality",\n                    status="failed",\n                    message="Potentially unterminated code block detected"\n                ))\n    \n    def _validate_consistency(self, file_path: Path, post) -> None:\n        """Validate content consistency"""\n        metadata = post.metadata\n        content = post.content\n        \n        # Check if title matches content\n        title = metadata.get('title', '')\n        first_heading = re.search(r'^#\\s+(.+)$', content, re.MULTILINE)\n        \n        if first_heading and title:\n            content_title = first_heading.group(1).strip()\n            if title.lower() != content_title.lower():\n                self.results.append(ValidationResult(\n                    file_path=str(file_path),\n                    validation_type="consistency",\n                    status="warning",\n                    message="Frontmatter title doesn't match first heading"\n                ))\n        \n        # Check filename consistency\n        expected_filename_parts = []\n        if 'date' in metadata:\n            date_str = metadata['date']\n            if isinstance(date_str, str):\n                expected_filename_parts.append(date_str)\n        \n        if title:\n            # Convert title to slug\n            slug = re.sub(r'[^\\w\\s-]', '', title.lower())\n            slug = re.sub(r'[\\s_-]+', '-', slug)\n            expected_filename_parts.append(slug[:50])  # Limit length\n        \n        if expected_filename_parts:\n            expected_filename = '-'.join(expected_filename_parts) + '.md'\n            actual_filename = file_path.name\n            \n            if actual_filename != expected_filename:\n                self.results.append(ValidationResult(\n                    file_path=str(file_path),\n                    validation_type="consistency",\n                    status="warning",\n                    message=f"Filename '{actual_filename}' doesn't follow expected pattern '{expected_filename}'"\n                ))\n    \n    def _validate_links(self, file_path: Path, post) -> None:\n        """Validate internal and external links"""\n        content = post.content\n        \n        # Find all markdown links\n        markdown_links = re.findall(r'\\[([^\\]]+)\\]\\(([^)]+)\\)', content)\n        \n        for link_text, link_url in markdown_links:\n            self._check_single_link(file_path, link_text, link_url)\n        \n        # Find all HTML links\n        html_links = re.findall(r'\u003Cspan class=\"language-xml\"><a[^>\u003C/span>]+href=[\u003Cspan class=\"hljs-string\">"\\'\u003C/span>](\u003Cspan class=\"hljs-link\">[^"\\']+\u003C/span>)[\u003Cspan class=\"hljs-string\">"\\'\u003C/span>][\u003Cspan class=\"hljs-symbol\">^>\u003C/span>]*\u003C/span>>([^<]\u003Cspan class=\"hljs-emphasis\">*)\u003Cspan class=\"language-xml\">\u003Cspan class=\"hljs-tag\"></\u003Cspan class=\"hljs-name\">a\u003C/span>>\u003C/span>\u003C/span>', content, re.IGNORECASE)\n        \n        for link_url, link_text in html_links:\n            self._check_single_link(file_path, link_text, link_url)\n    \n    def _check_single_link(self, file_path: Path, link_text: str, link_url: str) -> None:\n        """Check a single link for validity"""\n        # Skip anchor links and mailto links\n        if link_url.startswith('#') or link_url.startswith('mailto:'):\n            return\n        \n        # Check internal links\n        if link_url.startswith('/') or not urlparse(link_url).netloc:\n            # Internal link\n            if link_url.startswith('/'):\n                # Absolute internal link\n                target_path = self.content_dir / link_url.lstrip('/')\n            else:\n                # Relative link\n                target_path = file_path.parent / link_url\n            \n            # Check if file exists\n            if not target_path.exists():\n                # Try with .md extension\n                if not str(target_path).endswith('.md'):\n                    target_path_md = target_path.with_suffix('.md')\n                    if not target_path_md.exists():\n                        self.results.append(ValidationResult(\n                            file_path=str(file_path),\n                            validation_type="links",\n                            status="failed",\n                            message=f"Broken internal link: '{link_url}' -> '{link_text}'"\n                        ))\n                else:\n                    self.results.append(ValidationResult(\n                        file_path=str(file_path),\n                        validation_type="links",\n                        status="failed",\n                        message=f"Broken internal link: '{link_url}' -> '{link_text}'"\n                    ))\n        \n        else:\n            # External link - check if we should validate\n            if self._should_check_external_link(link_url):\n                self._check_external_link(file_path, link_text, link_url)\n    \n    def _should_check_external_link(self, url: str) -> bool:\n        """Determine if external link should be checked"""\n        # Skip certain domains that might block automated requests\n        skip_domains = [\n            'localhost',\n            '127.0.0.1',\n            'example.com',\n            'test.com'\n        ]\n        \n        parsed = urlparse(url)\n        return parsed.netloc not in skip_domains\n    \n    def _check_external_link(self, file_path: Path, link_text: str, link_url: str) -> None:\n        """Check external link availability"""\n        if link_url in self.link_cache:\n            # Use cached result\n            status = self.link_cache[link_url]\n        else:\n            try:\n                # Make HEAD request first (faster)\n                response = requests.head(link_url, timeout=10, allow_redirects=True)\n                status = response.status_code\n                \n                # If HEAD fails, try GET\n                if status >= 400:\n                    response = requests.get(link_url, timeout=10, allow_redirects=True)\n                    status = response.status_code\n                \n                self.link_cache[link_url] = status\n                \n            except requests.RequestException as e:\n                status = f"error: {str(e)}"\n                self.link_cache[link_url] = status\n        \n        if isinstance(status, int) and status >= 400:\n            self.results.append(ValidationResult(\n                file_path=str(file_path),\n                validation_type="links",\n                status="failed",\n                message=f"Broken external link (HTTP {status}): '{link_url}' -> '{link_text}'"\n            ))\n        elif isinstance(status, str) and status.startswith("error:"):\n            self.results.append(ValidationResult(\n                file_path=str(file_path),\n                validation_type="links",\n                status="warning",\n                message=f"External link check failed: '{link_url}' -> '{link_text}'"\n            ))\n    \n    def _validate_cross_file_consistency(self, files: List[Path]) -> None:\n        """Validate consistency across multiple files"""\n        # Check for duplicate titles\n        titles = {}\n        categories = set()\n        \n        for file_path in files:\n            try:\n                with open(file_path, 'r', encoding='utf-8') as f:\n                    post = frontmatter.load(f)\n                \n                title = post.metadata.get('title', '')\n                category = post.metadata.get('category', '')\n                \n                if title:\n                    if title in titles:\n                        self.results.append(ValidationResult(\n                            file_path=str(file_path),\n                            validation_type="cross_file_consistency",\n                            status="warning",\n                            message=f"Duplicate title found: '{title}' (also in {titles[title]})"\n                        ))\n                    else:\n                        titles[title] = str(file_path)\n                \n                if category:\n                    categories.add(category)\n                    \n            except Exception as e:\n                continue\n        \n        # Report category statistics\n        print(f"Found {len(categories)} unique categories: {', '.join(sorted(categories))}")\n    \n    def _calculate_stats(self) -> None:\n        """Calculate validation statistics"""\n        for result in self.results:\n            if result.status == "passed":\n                self.validation_stats['passed'] += 1\n            elif result.status == "failed":\n                self.validation_stats['failed'] += 1\n            elif result.status == "warning":\n                self.validation_stats['warnings'] += 1\n    \n    def generate_report(self, output_path: Optional[str] = None) -> Dict[str, Any]:\n        """Generate comprehensive validation report"""\n        report = {\n            'validation_summary': self.validation_stats,\n            'validation_timestamp': datetime.now().isoformat(),\n            'results_by_type': {},\n            'results_by_status': {},\n            'detailed_results': []\n        }\n        \n        # Group results by type and status\n        for result in self.results:\n            # By type\n            if result.validation_type not in report['results_by_type']:\n                report[\u003Cspan class=\"hljs-string\">'results_by_type'\u003C/span>][\u003Cspan class=\"hljs-symbol\">result.validation_type\u003C/span>] = []\n            report[\u003Cspan class=\"hljs-string\">'results_by_type'\u003C/span>][\u003Cspan class=\"hljs-symbol\">result.validation_type\u003C/span>].append({\n                'file': result.file_path,\n                'status': result.status,\n                'message': result.message\n            })\n            \n            # By status\n            if result.status not in report['results_by_status']:\n                report[\u003Cspan class=\"hljs-string\">'results_by_status'\u003C/span>][\u003Cspan class=\"hljs-symbol\">result.status\u003C/span>] = []\n            report[\u003Cspan class=\"hljs-string\">'results_by_status'\u003C/span>][\u003Cspan class=\"hljs-symbol\">result.status\u003C/span>].append({\n                'file': result.file_path,\n                'type': result.validation_type,\n                'message': result.message\n            })\n            \n            # Detailed results\n            report['detailed_results'].append({\n                'file_path': result.file_path,\n                'validation_type': result.validation_type,\n                'status': result.status,\n                'message': result.message,\n                'details': result.details\n            })\n        \n        if output_path:\n            with open(output_path, 'w') as f:\n                json.dump(report, f, indent=2)\n        \n        return report\n    \n    def print_summary(self) -> None:\n        """Print validation summary to console"""\n        stats = self.validation_stats\n        total_issues = stats['failed'] + stats['warnings']\n        \n        print(f"\\n=== Validation Summary ===")\n        print(f"Total files: {stats['total_files']}")\n        print(f"Issues found: {total_issues}")\n        print(f"  - Failed: {stats['failed']}")\n        print(f"  - Warnings: {stats['warnings']}")\n        \n        if total_issues == 0:\n            print("✅ All validations passed!")\n        else:\n            print(f"⚠️  {total_issues} issues need attention")\n            \n            # Show top issues by type\n            issue_types = {}\n            for result in self.results:\n                if result.status in ['failed', 'warning']:\n                    issue_types[result.validation_type] = issue_types.get(result.validation_type, 0) + 1\n            \n            print("\\nIssues by type:")\n            for issue_type, count in sorted(issue_types.items(), key=lambda x: x[1], reverse=True):\n                print(f"  - {issue_type}: {count}")\n\n# CLI Interface\ndef main():\n    import argparse\n    \n    parser = argparse.ArgumentParser(description='Validate generated content')\n    parser.add_argument('--content-dir', required=True, help='Directory containing content to validate')\n    parser.add_argument('--base-url', help='Base URL for link validation')\n    parser.add_argument('--check-links', action='store_true', help='Validate internal and external links')\n    parser.add_argument('--check-frontmatter', action='store_true', help='Validate frontmatter')\n    parser.add_argument('--check-content-quality', action='store_true', help='Check content quality')\n    parser.add_argument('--check-consistency', action='store_true', help='Check consistency')\n    parser.add_argument('--output', help='Output file for validation report')\n    parser.add_argument('--fail-on-error', action='store_true', help='Exit with error code if validation fails')\n    \n    args = parser.parse_args()\n    \n    validator = ContentValidator(args.content_dir, args.base_url or "")\n    \n    results = validator.validate_all_content(\n        check_links=args.check_links,\n        check_frontmatter=args.check_frontmatter,\n        check_content_quality=args.check_content_quality,\n        check_consistency=args.check_consistency\n    )\n    \n    # Generate and save report\n    report = validator.generate_report(args.output)\n    \n    # Print summary\n    validator.print_summary()\n    \n    # Exit with error code if requested and validation failed\n    if args.fail_on_error and validator.validation_stats['failed'] > 0:\n        exit(1)\n\nif \u003Cspan class=\"hljs-strong\">__name__\u003C/span> == "\u003Cspan class=\"hljs-strong\">__main__\u003C/span>":\n    main()\n```\u003C/span>\n\u003C/code>\u003C/pre>\u003Ch2 id=\"integration-with-documentation-systems\">Integration with Documentation Systems\u003C/h2>\n\u003Cp>Dynamic content generation integrates seamlessly with comprehensive documentation workflows. When combined with \u003Ca href=\"/blog/markdown-accessibility-complete-guide\">accessibility and WCAG compliance systems\u003C/a>, automated content generation ensures generated documentation maintains accessibility standards while producing consistent, high-quality content that serves users with diverse abilities and technology requirements.\u003C/p>\n\u003Cp>For comprehensive content management, automation systems work effectively with \u003Ca href=\"/blog/markdown-table-data-validation-and-quality-assurance-complete-guide\">table data validation and quality assurance frameworks\u003C/a> to ensure that dynamically generated tabular content maintains data integrity and formatting standards while preventing common table errors through automated validation and correction processes.\u003C/p>\n\u003Cp>When building sophisticated documentation platforms, content generation complements \u003Ca href=\"/blog/markdown-code-syntax-highlighting-advanced-techniques\">advanced code syntax highlighting systems\u003C/a> by enabling automated generation of code documentation with proper syntax highlighting, language detection, and interactive examples that enhance developer experience while maintaining consistency across large codebases.\u003C/p>\n\u003Ch2 id=\"advanced-automation-patterns\">Advanced Automation Patterns\u003C/h2>\n\u003Ch3 id=\"multi-source-content-aggregation\">Multi-Source Content Aggregation\u003C/h3>\n\u003Cp>Implementing sophisticated content aggregation from multiple data sources:\u003C/p>\n\u003Cpre>\u003Ccode class=\"hljs language-markdown\">\u003Cspan class=\"hljs-section\"># Advanced Content Aggregation Patterns\u003C/span>\n\n\u003Cspan class=\"hljs-section\">## Multi-API Content Fusion\u003C/span>\n\u003Cspan class=\"hljs-code\">```python\n#!/usr/bin/env python3\n# multi-source-aggregator.py - Advanced content aggregation system\n\nimport asyncio\nimport aiohttp\nfrom typing import Dict, List, Any, Optional\nfrom dataclasses import dataclass, field\nimport json\nimport yaml\nfrom datetime import datetime, timedelta\nimport logging\nfrom jinja2 import Environment, DictLoader\nimport hashlib\n\n@dataclass\nclass DataSourceConfig:\n    """Configuration for individual data source"""\n    name: str\n    type: str\n    endpoint: str\n    auth_method: str = "none"\n    auth_config: Dict[str, Any] = field(default_factory=dict)\n    transform_pipeline: List[str] = field(default_factory=list)\n    cache_ttl: int = 3600\n    retry_config: Dict[str, Any] = field(default_factory=lambda: {"max_retries": 3, "delay": 1})\n\n@dataclass\nclass ContentTemplate:\n    """Template for generating aggregated content"""\n    name: str\n    template_string: str\n    output_path: str\n    data_sources: List[str]\n    aggregation_rules: Dict[str, Any] = field(default_factory=dict)\n    update_frequency: str = "hourly"\n\nclass MultiSourceContentAggregator:\n    """Advanced content aggregator supporting multiple data sources"""\n    \n    def __init__(self, config_path: str):\n        with open(config_path, 'r') as f:\n            self.config = yaml.safe_load(f)\n        \n        self.data_sources = {\n            ds['name']: DataSourceConfig(**ds)\n            for ds in self.config['data_sources']\n        }\n        \n        self.templates = {\n            tmpl['name']: ContentTemplate(**tmpl)\n            for tmpl in self.config['templates']\n        }\n        \n        self.data_cache = {}\n        self.cache_timestamps = {}\n        self.session = None\n        \n        # Initialize Jinja2 environment\n        self.jinja_env = Environment(loader=DictLoader({}))\n        self._setup_custom_filters()\n        \n        logging.basicConfig(level=logging.INFO)\n        self.logger = logging.getLogger(__name__)\n    \n    async def __aenter__(self):\n        """Async context manager entry"""\n        self.session = aiohttp.ClientSession()\n        return self\n    \n    async def __aexit__(self, exc_type, exc_val, exc_tb):\n        """Async context manager exit"""\n        if self.session:\n            await self.session.close()\n    \n    def _setup_custom_filters(self):\n        """Setup custom Jinja2 filters for content aggregation"""\n        \n        def merge_lists_filter(list1: List[Any], list2: List[Any], key: str = 'id') -> List[Any]:\n            """Merge two lists based on a key"""\n            result = list1.copy()\n            existing_keys = {item.get(key) for item in list1}\n            \n            for item in list2:\n                if item.get(key) not in existing_keys:\n                    result.append(item)\n            \n            return result\n        \n        def group_by_filter(items: List[Dict], key: str) -> Dict[str, List[Dict]]:\n            """Group list items by a key"""\n            grouped = {}\n            for item in items:\n                group_key = item.get(key, 'unknown')\n                if group_key not in grouped:\n                    grouped[group_key] = []\n                grouped[group_key].append(item)\n            return grouped\n        \n        def sort_by_priority_filter(items: List[Dict], priority_key: str = 'priority') -> List[Dict]:\n            """Sort items by priority (higher numbers first)"""\n            return sorted(items, key=lambda x: x.get(priority_key, 0), reverse=True)\n        \n        def calculate_metrics_filter(items: List[Dict], metric_key: str) -> Dict[str, Any]:\n            """Calculate aggregate metrics for a list of items"""\n            values = [item.get(metric_key, 0) for item in items if isinstance(item.get(metric_key), (int, float))]\n            \n            if not values:\n                return {'count': 0, 'sum': 0, 'avg': 0, 'min': 0, 'max': 0}\n            \n            return {\n                'count': len(values),\n                'sum': sum(values),\n                'avg': sum(values) / len(values),\n                'min': min(values),\n                'max': max(values)\n            }\n        \n        def filter_recent_filter(items: List[Dict], date_key: str, days: int = 7) -> List[Dict]:\n            """Filter items to only include recent entries"""\n            cutoff_date = datetime.now() - timedelta(days=days)\n            \n            recent_items = []\n            for item in items:\n                item_date = item.get(date_key)\n                if item_date:\n                    try:\n                        if isinstance(item_date, str):\n                            item_date = datetime.fromisoformat(item_date.replace('Z', '+00:00'))\n                        if item_date >= cutoff_date:\n                            recent_items.append(item)\n                    except ValueError:\n                        continue\n            \n            return recent_items\n        \n        # Register filters\n        self.jinja_env.filters['merge_lists'] = merge_lists_filter\n        self.jinja_env.filters['group_by'] = group_by_filter\n        self.jinja_env.filters['sort_by_priority'] = sort_by_priority_filter\n        self.jinja_env.filters['calculate_metrics'] = calculate_metrics_filter\n        self.jinja_env.filters['filter_recent'] = filter_recent_filter\n    \n    async def fetch_all_data(self, force_refresh: bool = False) -> Dict[str, Any]:\n        """Fetch data from all configured sources"""\n        tasks = []\n        \n        for source_name, source_config in self.data_sources.items():\n            if force_refresh or self._is_cache_stale(source_name, source_config.cache_ttl):\n                task = asyncio.create_task(\n                    self._fetch_source_data(source_name, source_config)\n                )\n                tasks.append(task)\n        \n        if tasks:\n            results = await asyncio.gather(*tasks, return_exceptions=True)\n            \n            for i, result in enumerate(results):\n                if isinstance(result, Exception):\n                    self.logger.error(f"Error fetching data: {result}")\n        \n        return self.data_cache\n    \n    def _is_cache_stale(self, source_name: str, ttl: int) -> bool:\n        """Check if cached data is stale"""\n        if source_name not in self.cache_timestamps:\n            return True\n        \n        age = datetime.now().timestamp() - self.cache_timestamps[source_name]\n        return age > ttl\n    \n    async def _fetch_source_data(self, source_name: str, config: DataSourceConfig) -> None:\n        """Fetch data from a single source with retry logic"""\n        for attempt in range(config.retry_config.get('max_retries', 3)):\n            try:\n                if config.type == 'rest_api':\n                    data = await self._fetch_rest_api_data(config)\n                elif config.type == 'graphql':\n                    data = await self._fetch_graphql_data(config)\n                elif config.type == 'database':\n                    data = await self._fetch_database_data(config)\n                elif config.type == 'file':\n                    data = await self._fetch_file_data(config)\n                else:\n                    raise ValueError(f"Unknown data source type: {config.type}")\n                \n                # Apply transformation pipeline\n                for transform in config.transform_pipeline:\n                    data = await self._apply_transformation(data, transform)\n                \n                self.data_cache[source_name] = data\n                self.cache_timestamps[source_name] = datetime.now().timestamp()\n                self.logger.info(f"Successfully fetched data from {source_name}")\n                return\n                \n            except Exception as e:\n                self.logger.warning(f"Attempt {attempt + 1} failed for {source_name}: {e}")\n                if attempt < config.retry_config.get('max_retries', 3) - 1:\n                    await asyncio.sleep(config.retry_config.get('delay', 1) * (2 ** attempt))\n                else:\n                    self.logger.error(f"Failed to fetch data from {source_name} after all retries")\n                    raise\n    \n    async def _fetch_rest_api_data(self, config: DataSourceConfig) -> Dict[str, Any]:\n        """Fetch data from REST API"""\n        headers = {}\n        \n        # Handle authentication\n        if config.auth_method == 'bearer_token':\n            headers['Authorization'] = f"Bearer {config.auth_config['token']}"\n        elif config.auth_method == 'api_key':\n            if config.auth_config.get('header_name'):\n                headers[config.auth_config['header_name']] = config.auth_config['api_key']\n            else:\n                headers['X-API-Key'] = config.auth_config['api_key']\n        elif config.auth_method == 'basic_auth':\n            import base64\n            credentials = f"{config.auth_config['username']}:{config.auth_config['password']}"\n            encoded = base64.b64encode(credentials.encode()).decode()\n            headers['Authorization'] = f"Basic {encoded}"\n        \n        async with self.session.get(config.endpoint, headers=headers) as response:\n            response.raise_for_status()\n            return await response.json()\n    \n    async def _fetch_graphql_data(self, config: DataSourceConfig) -> Dict[str, Any]:\n        """Fetch data from GraphQL endpoint"""\n        headers = {'Content-Type': 'application/json'}\n        \n        # Handle authentication\n        if config.auth_method == 'bearer_token':\n            headers['Authorization'] = f"Bearer {config.auth_config['token']}"\n        \n        query = config.auth_config.get('query', '')\n        variables = config.auth_config.get('variables', {})\n        \n        payload = {\n            'query': query,\n            'variables': variables\n        }\n        \n        async with self.session.post(config.endpoint, json=payload, headers=headers) as response:\n            response.raise_for_status()\n            result = await response.json()\n            \n            if 'errors' in result:\n                raise Exception(f"GraphQL errors: {result['errors']}")\n            \n            return result.get('data', {})\n    \n    async def _fetch_database_data(self, config: DataSourceConfig) -> Dict[str, Any]:\n        """Fetch data from database"""\n        # This would implement database connectivity\n        # For this example, we'll simulate with a placeholder\n        return {\n            'message': 'Database connectivity would be implemented here',\n            'config': config.endpoint\n        }\n    \n    async def _fetch_file_data(self, config: DataSourceConfig) -> Dict[str, Any]:\n        """Fetch data from file"""\n        if config.endpoint.startswith('http'):\n            async with self.session.get(config.endpoint) as response:\n                response.raise_for_status()\n                \n                if config.endpoint.endswith('.json'):\n                    return await response.json()\n                elif config.endpoint.endswith('.yaml') or config.endpoint.endswith('.yml'):\n                    text = await response.text()\n                    return yaml.safe_load(text)\n                else:\n                    return {'content': await response.text()}\n        else:\n            # Local file\n            with open(config.endpoint, 'r') as f:\n                if config.endpoint.endswith('.json'):\n                    return json.load(f)\n                elif config.endpoint.endswith('.yaml') or config.endpoint.endswith('.yml'):\n                    return yaml.safe_load(f)\n                else:\n                    return {'content': f.read()}\n    \n    async def _apply_transformation(self, data: Dict[str, Any], transform: str) -> Dict[str, Any]:\n        """Apply data transformation"""\n        # Simple transformations - in production, this would be more sophisticated\n        if transform == 'normalize_dates':\n            return self._normalize_dates(data)\n        elif transform == 'extract_summary':\n            return self._extract_summary(data)\n        elif transform == 'flatten_nested':\n            return self._flatten_nested_data(data)\n        elif transform == 'calculate_derived_metrics':\n            return self._calculate_derived_metrics(data)\n        \n        return data\n    \n    def _normalize_dates(self, data: Dict[str, Any]) -> Dict[str, Any]:\n        """Normalize date formats in data"""\n        def normalize_date_value(value):\n            if isinstance(value, str):\n                # Try to parse and normalize date strings\n                for fmt in ['%Y-%m-%dT%H:%M:%SZ', '%Y-%m-%d %H:%M:%S', '%Y-%m-%d']:\n                    try:\n                        dt = datetime.strptime(value, fmt)\n                        return dt.isoformat()\n                    except ValueError:\n                        continue\n            return value\n        \n        def normalize_recursive(obj):\n            if isinstance(obj, dict):\n                return {k: normalize_recursive(v) for k, v in obj.items()}\n            elif isinstance(obj, list):\n                return [normalize_recursive(item) for item in obj]\n            else:\n                return normalize_date_value(obj)\n        \n        return normalize_recursive(data)\n    \n    def _extract_summary(self, data: Dict[str, Any]) -> Dict[str, Any]:\n        """Extract summary information from data"""\n        summary = {\n            'total_items': 0,\n            'categories': set(),\n            'date_range': {'earliest': None, 'latest': None},\n            'data_types': set()\n        }\n        \n        def analyze_item(item, path=""):\n            if isinstance(item, dict):\n                for key, value in item.items():\n                    new_path = f"{path}.{key}" if path else key\n                    analyze_item(value, new_path)\n                    \n                    if 'category' in key.lower():\n                        summary['categories'].add(str(value))\n            elif isinstance(item, list):\n                summary['total_items'] += len(item)\n                for subitem in item:\n                    analyze_item(subitem, path)\n            else:\n                summary['data_types'].add(type(item).__name__)\n        \n        analyze_item(data)\n        \n        # Convert sets to lists for JSON serialization\n        data['_summary'] = {\n            'total_items': summary['total_items'],\n            'categories': list(summary['categories']),\n            'data_types': list(summary['data_types'])\n        }\n        \n        return data\n    \n    def _flatten_nested_data(self, data: Dict[str, Any]) -> Dict[str, Any]:\n        """Flatten deeply nested data structures"""\n        def flatten_dict(d, parent_key='', sep='_'):\n            items = []\n            for k, v in d.items():\n                new_key = f"{parent_key}{sep}{k}" if parent_key else k\n                if isinstance(v, dict):\n                    items.extend(flatten_dict(v, new_key, sep=sep).items())\n                else:\n                    items.append((new_key, v))\n            return dict(items)\n        \n        if isinstance(data, dict):\n            data['_flattened'] = flatten_dict(data)\n        \n        return data\n    \n    def _calculate_derived_metrics(self, data: Dict[str, Any]) -> Dict[str, Any]:\n        """Calculate derived metrics from raw data"""\n        metrics = {}\n        \n        # Look for numeric fields to calculate aggregations\n        def find_numeric_fields(obj, path=""):\n            if isinstance(obj, dict):\n                for key, value in obj.items():\n                    new_path = f"{path}.{key}" if path else key\n                    if isinstance(value, (int, float)):\n                        if 'metrics' not in data:\n                            data['metrics'] = {}\n                        if path not in data['metrics']:\n                            data['metrics'][path] = []\n                        data['metrics'][path].append(value)\n                    elif isinstance(value, (dict, list)):\n                        find_numeric_fields(value, new_path)\n            elif isinstance(obj, list):\n                for item in obj:\n                    find_numeric_fields(item, path)\n        \n        find_numeric_fields(data)\n        \n        # Calculate aggregations\n        if 'metrics' in data:\n            for field_path, values in data['metrics'].items():\n                if values:\n                    metrics[field_path] = {\n                        'count': len(values),\n                        'sum': sum(values),\n                        'avg': sum(values) / len(values),\n                        'min': min(values),\n                        'max': max(values)\n                    }\n            \n            data['_derived_metrics'] = metrics\n        \n        return data\n    \n    async def generate_aggregated_content(self, template_name: str) -> str:\n        """Generate content using aggregated data from multiple sources"""\n        if template_name not in self.templates:\n            raise ValueError(f"Template '{template_name}' not found")\n        \n        template_config = self.templates[template_name]\n        \n        # Fetch required data sources\n        required_data = {}\n        for source_name in template_config.data_sources:\n            if source_name not in self.data_cache or self._is_cache_stale(\n                source_name, self.data_sources[source_name].cache_ttl\n            ):\n                await self._fetch_source_data(source_name, self.data_sources[source_name])\n            \n            required_data[source_name] = self.data_cache[source_name]\n        \n        # Apply aggregation rules\n        aggregated_data = await self._apply_aggregation_rules(required_data, template_config.aggregation_rules)\n        \n        # Create template context\n        context = {\n            'data': aggregated_data,\n            'sources': required_data,\n            'config': self.config,\n            'generation_time': datetime.now(),\n            'template_name': template_name\n        }\n        \n        # Render template\n        template = self.jinja_env.from_string(template_config.template_string)\n        content = template.render(**context)\n        \n        # Write output file\n        output_path = template_config.output_path\n        os.makedirs(os.path.dirname(output_path), exist_ok=True)\n        \n        with open(output_path, 'w', encoding='utf-8') as f:\n            f.write(content)\n        \n        self.logger.info(f"Generated aggregated content: {output_path}")\n        return content\n    \n    async def _apply_aggregation_rules(self, data: Dict[str, Any], rules: Dict[str, Any]) -> Dict[str, Any]:\n        """Apply aggregation rules to combine data from multiple sources"""\n        result = {}\n        \n        for rule_name, rule_config in rules.items():\n            rule_type = rule_config.get('type', 'merge')\n            \n            if rule_type == 'merge':\n                # Merge data from specified sources\n                sources = rule_config.get('sources', [])\n                merged_data = {}\n                \n                for source in sources:\n                    if source in data:\n                        merged_data.update(data[source])\n                \n                result[rule_name] = merged_data\n            \n            elif rule_type == 'combine_lists':\n                # Combine lists from multiple sources\n                sources = rule_config.get('sources', [])\n                list_key = rule_config.get('list_key', 'items')\n                combined_list = []\n                \n                for source in sources:\n                    if source in data and list_key in data[source]:\n                        source_items = data[source][list_key]\n                        if isinstance(source_items, list):\n                            # Add source identifier to each item\n                            for item in source_items:\n                                if isinstance(item, dict):\n                                    item['_source'] = source\n                            combined_list.extend(source_items)\n                \n                result[rule_name] = combined_list\n            \n            elif rule_type == 'cross_reference':\n                # Cross-reference data between sources\n                primary_source = rule_config.get('primary_source')\n                reference_source = rule_config.get('reference_source')\n                join_key = rule_config.get('join_key', 'id')\n                \n                if primary_source in data and reference_source in data:\n                    primary_data = data[primary_source]\n                    reference_data = data[reference_source]\n                    \n                    # Create lookup table for reference data\n                    ref_lookup = {}\n                    if isinstance(reference_data, dict) and 'items' in reference_data:\n                        for item in reference_data['items']:\n                            if isinstance(item, dict) and join_key in item:\n                                ref_lookup[item[join_key]] = item\n                    \n                    # Enhance primary data with reference information\n                    enhanced_data = primary_data.copy()\n                    if isinstance(enhanced_data, dict) and 'items' in enhanced_data:\n                        for item in enhanced_data['items']:\n                            if isinstance(item, dict) and join_key in item:\n                                ref_item = ref_lookup.get(item[join_key])\n                                if ref_item:\n                                    item['_reference'] = ref_item\n                    \n                    result[rule_name] = enhanced_data\n        \n        return result\n\n# Example usage configuration\nexample_config = """\ndata_sources:\n  - name: "github_issues"\n    type: "rest_api"\n    endpoint: "https://api.github.com/repos/owner/repo/issues"\n    auth_method: "bearer_token"\n    auth_config:\n      token: "${GITHUB_TOKEN}"\n    transform_pipeline:\n      - "normalize_dates"\n      - "extract_summary"\n    cache_ttl: 1800\n    \n  - name: "user_metrics"\n    type: "rest_api"\n    endpoint: "https://analytics.example.com/api/users/metrics"\n    auth_method: "api_key"\n    auth_config:\n      api_key: "${ANALYTICS_API_KEY}"\n      header_name: "X-Analytics-Key"\n    transform_pipeline:\n      - "calculate_derived_metrics"\n    cache_ttl: 3600\n    \n  - name: "feature_flags"\n    type: "file"\n    endpoint: "config/features.json"\n    transform_pipeline:\n      - "extract_summary"\n    cache_ttl: 600\n\ntemplates:\n  - name: "status_dashboard"\n    output_path: "docs/status/dashboard.md"\n    data_sources: ["github_issues", "user_metrics", "feature_flags"]\n    aggregation_rules:\n      recent_activity:\n        type: "combine_lists"\n        sources: ["github_issues"]\n        list_key: "items"\n      metrics_summary:\n        type: "merge"\n        sources: ["user_metrics"]\n      feature_status:\n        type: "cross_reference"\n        primary_source: "feature_flags"\n        reference_source: "user_metrics"\n        join_key: "feature_id"\n    template_string: |\n      \n      ---\n      title: "System Status Dashboard"\n      description: "Real-time system status and metrics overview"\n      category: "Status"\n      date: {{ generation_time.strftime('%Y-%m-%d') }}\n      ---\n      \n      # System Status Dashboard\n      \n      *Last updated: {{ generation_time.strftime('%Y-%m-%d %H:%M:%S') }} UTC*\n      \n      ## Recent Activity\n      \n      {% set recent_issues = data.recent_activity | filter_recent('created_at', 7) | sort_by_priority('priority') %}\n      \n      ### Open Issues (Last 7 Days)\n      | Issue | Priority | Status | Created |\n      |:------|:---------|:-------|:--------|\n      {% for issue in recent_issues[:10] %}\n      | [#{{ issue.number }}]({{ issue.html_url }}) {{ issue.title }} | {{ issue.labels | selectattr('name', 'match', 'priority:.*') | map(attribute='name') | join(', ') | default('normal') }} | {{ issue.state }} | {{ issue.created_at | relative_time }} |\n      {% endfor %}\n      \n      ## System Metrics\n      \n      {% set metrics = data.metrics_summary %}\n      \n      ### Performance Overview\n      - **Active Users**: {{ metrics.active_users | format_number('abbreviated') }}\n      - **Response Time**: {{ metrics.avg_response_time }}ms\n      - **Error Rate**: {{ metrics.error_rate }}%\n      - **Uptime**: {{ metrics.uptime_percentage }}%\n      \n      ### Usage Statistics\n      {% for category, stats in metrics.usage_by_category.items() %}\n      - **{{ category }}**: {{ stats.requests | format_number('abbreviated') }} requests\n      {% endfor %}\n      \n      ## Feature Status\n      \n      {% set features = data.feature_status %}\n      \n      ### Feature Rollout Status\n      {% for feature in features.items | group_by('status').items() %}\n      \n      #### {{ feature[0] | title }} Features\n      {% for item in feature[1] %}\n      - **{{ item.name }}**: {{ item.rollout_percentage }}% rollout\n        {% if item._reference %}\n        - Usage: {{ item._reference.usage_count | format_number('abbreviated') }} \n        {% endif %}\n      {% endfor %}\n      {% endfor %}\n      \n      ---\n      *This dashboard is automatically generated from multiple data sources and updated every hour.*\n      \n    update_frequency: "hourly"\n"""\n\n# CLI interface for the aggregator\nasync def main():\n    import argparse\n    import os\n    \n    parser = argparse.ArgumentParser(description='Multi-source content aggregator')\n    parser.add_argument('--config', required=True, help='Configuration file path')\n    parser.add_argument('--template', help='Specific template to generate')\n    parser.add_argument('--force-refresh', action='store_true', help='Force refresh of all data sources')\n    \n    args = parser.parse_args()\n    \n    async with MultiSourceContentAggregator(args.config) as aggregator:\n        # Fetch all data\n        await aggregator.fetch_all_data(force_refresh=args.force_refresh)\n        \n        if args.template:\n            # Generate specific template\n            await aggregator.generate_aggregated_content(args.template)\n            print(f"Generated content for template: {args.template}")\n        else:\n            # Generate all templates\n            for template_name in aggregator.templates:\n                try:\n                    await aggregator.generate_aggregated_content(template_name)\n                    print(f"✅ Generated: {template_name}")\n                except Exception as e:\n                    print(f"❌ Failed to generate {template_name}: {e}")\n\nif __name__ == "__main__":\n    asyncio.run(main())\n```\u003C/span>\n\u003C/code>\u003C/pre>\u003Ch2 id=\"performance-optimization-and-scaling\">Performance Optimization and Scaling\u003C/h2>\n\u003Ch3 id=\"content-generation-performance-monitoring\">Content Generation Performance Monitoring\u003C/h3>\n\u003Cp>Implementing comprehensive performance monitoring for large-scale content generation:\u003C/p>\n\u003Cpre>\u003Ccode class=\"hljs language-python\">\u003Cspan class=\"hljs-comment\">#!/usr/bin/env python3\u003C/span>\n\u003Cspan class=\"hljs-comment\"># performance-monitor.py - Performance monitoring for content generation\u003C/span>\n\n\u003Cspan class=\"hljs-keyword\">import\u003C/span> time\n\u003Cspan class=\"hljs-keyword\">import\u003C/span> psutil\n\u003Cspan class=\"hljs-keyword\">import\u003C/span> threading\n\u003Cspan class=\"hljs-keyword\">from\u003C/span> typing \u003Cspan class=\"hljs-keyword\">import\u003C/span> \u003Cspan class=\"hljs-type\">Dict\u003C/span>, \u003Cspan class=\"hljs-type\">List\u003C/span>, \u003Cspan class=\"hljs-type\">Any\u003C/span>, \u003Cspan class=\"hljs-type\">Optional\u003C/span>\n\u003Cspan class=\"hljs-keyword\">from\u003C/span> dataclasses \u003Cspan class=\"hljs-keyword\">import\u003C/span> dataclass, field\n\u003Cspan class=\"hljs-keyword\">from\u003C/span> datetime \u003Cspan class=\"hljs-keyword\">import\u003C/span> datetime, timedelta\n\u003Cspan class=\"hljs-keyword\">import\u003C/span> statistics\n\u003Cspan class=\"hljs-keyword\">import\u003C/span> json\n\u003Cspan class=\"hljs-keyword\">import\u003C/span> asyncio\n\u003Cspan class=\"hljs-keyword\">from\u003C/span> concurrent.futures \u003Cspan class=\"hljs-keyword\">import\u003C/span> ThreadPoolExecutor, ProcessPoolExecutor\n\u003Cspan class=\"hljs-keyword\">import\u003C/span> multiprocessing \u003Cspan class=\"hljs-keyword\">as\u003C/span> mp\n\n\u003Cspan class=\"hljs-meta\">@dataclass\u003C/span>\n\u003Cspan class=\"hljs-keyword\">class\u003C/span> \u003Cspan class=\"hljs-title class_\">PerformanceMetrics\u003C/span>:\n    \u003Cspan class=\"hljs-string\">"""Performance metrics for content generation operations"""\u003C/span>\n    operation_id: \u003Cspan class=\"hljs-built_in\">str\u003C/span>\n    operation_type: \u003Cspan class=\"hljs-built_in\">str\u003C/span>\n    start_time: datetime\n    end_time: \u003Cspan class=\"hljs-type\">Optional\u003C/span>[datetime] = \u003Cspan class=\"hljs-literal\">None\u003C/span>\n    duration_seconds: \u003Cspan class=\"hljs-type\">Optional\u003C/span>[\u003Cspan class=\"hljs-built_in\">float\u003C/span>] = \u003Cspan class=\"hljs-literal\">None\u003C/span>\n    memory_usage_mb: \u003Cspan class=\"hljs-type\">Optional\u003C/span>[\u003Cspan class=\"hljs-built_in\">float\u003C/span>] = \u003Cspan class=\"hljs-literal\">None\u003C/span>\n    cpu_usage_percent: \u003Cspan class=\"hljs-type\">Optional\u003C/span>[\u003Cspan class=\"hljs-built_in\">float\u003C/span>] = \u003Cspan class=\"hljs-literal\">None\u003C/span>\n    files_processed: \u003Cspan class=\"hljs-type\">Optional\u003C/span>[\u003Cspan class=\"hljs-built_in\">int\u003C/span>] = \u003Cspan class=\"hljs-literal\">None\u003C/span>\n    output_size_kb: \u003Cspan class=\"hljs-type\">Optional\u003C/span>[\u003Cspan class=\"hljs-built_in\">float\u003C/span>] = \u003Cspan class=\"hljs-literal\">None\u003C/span>\n    cache_hits: \u003Cspan class=\"hljs-type\">Optional\u003C/span>[\u003Cspan class=\"hljs-built_in\">int\u003C/span>] = \u003Cspan class=\"hljs-literal\">None\u003C/span>\n    cache_misses: \u003Cspan class=\"hljs-type\">Optional\u003C/span>[\u003Cspan class=\"hljs-built_in\">int\u003C/span>] = \u003Cspan class=\"hljs-literal\">None\u003C/span>\n\n\u003Cspan class=\"hljs-keyword\">class\u003C/span> \u003Cspan class=\"hljs-title class_\">ContentGenerationPerformanceMonitor\u003C/span>:\n    \u003Cspan class=\"hljs-string\">"""Performance monitoring system for content generation"""\u003C/span>\n    \n    \u003Cspan class=\"hljs-keyword\">def\u003C/span> \u003Cspan class=\"hljs-title function_\">__init__\u003C/span>(\u003Cspan class=\"hljs-params\">self, enable_detailed_monitoring: \u003Cspan class=\"hljs-built_in\">bool\u003C/span> = \u003Cspan class=\"hljs-literal\">True\u003C/span>\u003C/span>):\n        self.enable_detailed_monitoring = enable_detailed_monitoring\n        self.metrics_history: \u003Cspan class=\"hljs-type\">List\u003C/span>[PerformanceMetrics] = []\n        self.active_operations: \u003Cspan class=\"hljs-type\">Dict\u003C/span>[\u003Cspan class=\"hljs-built_in\">str\u003C/span>, PerformanceMetrics] = {}\n        self.system_metrics = []\n        \n        \u003Cspan class=\"hljs-comment\"># Start system monitoring thread\u003C/span>\n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> enable_detailed_monitoring:\n            self._start_system_monitoring()\n    \n    \u003Cspan class=\"hljs-keyword\">def\u003C/span> \u003Cspan class=\"hljs-title function_\">_start_system_monitoring\u003C/span>(\u003Cspan class=\"hljs-params\">self\u003C/span>):\n        \u003Cspan class=\"hljs-string\">"""Start background system monitoring"""\u003C/span>\n        \u003Cspan class=\"hljs-keyword\">def\u003C/span> \u003Cspan class=\"hljs-title function_\">monitor_system\u003C/span>():\n            \u003Cspan class=\"hljs-keyword\">while\u003C/span> \u003Cspan class=\"hljs-literal\">True\u003C/span>:\n                \u003Cspan class=\"hljs-keyword\">try\u003C/span>:\n                    cpu_percent = psutil.cpu_percent(interval=\u003Cspan class=\"hljs-number\">1\u003C/span>)\n                    memory = psutil.virtual_memory()\n                    disk = psutil.disk_usage(\u003Cspan class=\"hljs-string\">'/'\u003C/span>)\n                    \n                    self.system_metrics.append({\n                        \u003Cspan class=\"hljs-string\">'timestamp'\u003C/span>: datetime.now(),\n                        \u003Cspan class=\"hljs-string\">'cpu_percent'\u003C/span>: cpu_percent,\n                        \u003Cspan class=\"hljs-string\">'memory_percent'\u003C/span>: memory.percent,\n                        \u003Cspan class=\"hljs-string\">'memory_available_gb'\u003C/span>: memory.available / (\u003Cspan class=\"hljs-number\">1024\u003C/span>**\u003Cspan class=\"hljs-number\">3\u003C/span>),\n                        \u003Cspan class=\"hljs-string\">'disk_free_gb'\u003C/span>: disk.free / (\u003Cspan class=\"hljs-number\">1024\u003C/span>**\u003Cspan class=\"hljs-number\">3\u003C/span>)\n                    })\n                    \n                    \u003Cspan class=\"hljs-comment\"># Keep only last hour of system metrics\u003C/span>\n                    cutoff = datetime.now() - timedelta(hours=\u003Cspan class=\"hljs-number\">1\u003C/span>)\n                    self.system_metrics = [\n                        m \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> self.system_metrics \n                        \u003Cspan class=\"hljs-keyword\">if\u003C/span> m[\u003Cspan class=\"hljs-string\">'timestamp'\u003C/span>] > cutoff\n                    ]\n                    \n                    time.sleep(\u003Cspan class=\"hljs-number\">30\u003C/span>)  \u003Cspan class=\"hljs-comment\"># Sample every 30 seconds\u003C/span>\n                    \n                \u003Cspan class=\"hljs-keyword\">except\u003C/span> Exception \u003Cspan class=\"hljs-keyword\">as\u003C/span> e:\n                    \u003Cspan class=\"hljs-built_in\">print\u003C/span>(\u003Cspan class=\"hljs-string\">f"System monitoring error: \u003Cspan class=\"hljs-subst\">{e}\u003C/span>"\u003C/span>)\n                    time.sleep(\u003Cspan class=\"hljs-number\">60\u003C/span>)\n        \n        monitor_thread = threading.Thread(target=monitor_system, daemon=\u003Cspan class=\"hljs-literal\">True\u003C/span>)\n        monitor_thread.start()\n    \n    \u003Cspan class=\"hljs-keyword\">def\u003C/span> \u003Cspan class=\"hljs-title function_\">start_operation\u003C/span>(\u003Cspan class=\"hljs-params\">self, operation_id: \u003Cspan class=\"hljs-built_in\">str\u003C/span>, operation_type: \u003Cspan class=\"hljs-built_in\">str\u003C/span>\u003C/span>) -> PerformanceMetrics:\n        \u003Cspan class=\"hljs-string\">"""Start monitoring an operation"""\u003C/span>\n        metrics = PerformanceMetrics(\n            operation_id=operation_id,\n            operation_type=operation_type,\n            start_time=datetime.now()\n        )\n        \n        self.active_operations[operation_id] = metrics\n        \u003Cspan class=\"hljs-keyword\">return\u003C/span> metrics\n    \n    \u003Cspan class=\"hljs-keyword\">def\u003C/span> \u003Cspan class=\"hljs-title function_\">end_operation\u003C/span>(\u003Cspan class=\"hljs-params\">self, operation_id: \u003Cspan class=\"hljs-built_in\">str\u003C/span>, \n                     files_processed: \u003Cspan class=\"hljs-built_in\">int\u003C/span> = \u003Cspan class=\"hljs-number\">0\u003C/span>,\n                     output_size_kb: \u003Cspan class=\"hljs-built_in\">float\u003C/span> = \u003Cspan class=\"hljs-number\">0\u003C/span>,\n                     cache_hits: \u003Cspan class=\"hljs-built_in\">int\u003C/span> = \u003Cspan class=\"hljs-number\">0\u003C/span>,\n                     cache_misses: \u003Cspan class=\"hljs-built_in\">int\u003C/span> = \u003Cspan class=\"hljs-number\">0\u003C/span>\u003C/span>) -> PerformanceMetrics:\n        \u003Cspan class=\"hljs-string\">"""End monitoring and record final metrics"""\u003C/span>\n        \n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> operation_id \u003Cspan class=\"hljs-keyword\">not\u003C/span> \u003Cspan class=\"hljs-keyword\">in\u003C/span> self.active_operations:\n            \u003Cspan class=\"hljs-keyword\">return\u003C/span> \u003Cspan class=\"hljs-literal\">None\u003C/span>\n        \n        metrics = self.active_operations[operation_id]\n        metrics.end_time = datetime.now()\n        metrics.duration_seconds = (metrics.end_time - metrics.start_time).total_seconds()\n        metrics.files_processed = files_processed\n        metrics.output_size_kb = output_size_kb\n        metrics.cache_hits = cache_hits\n        metrics.cache_misses = cache_misses\n        \n        \u003Cspan class=\"hljs-comment\"># Capture resource usage\u003C/span>\n        \u003Cspan class=\"hljs-keyword\">try\u003C/span>:\n            process = psutil.Process()\n            memory_info = process.memory_info()\n            metrics.memory_usage_mb = memory_info.rss / (\u003Cspan class=\"hljs-number\">1024\u003C/span> * \u003Cspan class=\"hljs-number\">1024\u003C/span>)\n            metrics.cpu_usage_percent = process.cpu_percent()\n        \u003Cspan class=\"hljs-keyword\">except\u003C/span> Exception:\n            \u003Cspan class=\"hljs-keyword\">pass\u003C/span>\n        \n        \u003Cspan class=\"hljs-comment\"># Move to history\u003C/span>\n        self.metrics_history.append(metrics)\n        \u003Cspan class=\"hljs-keyword\">del\u003C/span> self.active_operations[operation_id]\n        \n        \u003Cspan class=\"hljs-keyword\">return\u003C/span> metrics\n    \n    \u003Cspan class=\"hljs-keyword\">def\u003C/span> \u003Cspan class=\"hljs-title function_\">get_performance_summary\u003C/span>(\u003Cspan class=\"hljs-params\">self, \n                              operation_type: \u003Cspan class=\"hljs-type\">Optional\u003C/span>[\u003Cspan class=\"hljs-built_in\">str\u003C/span>] = \u003Cspan class=\"hljs-literal\">None\u003C/span>,\n                              time_window: \u003Cspan class=\"hljs-type\">Optional\u003C/span>[timedelta] = \u003Cspan class=\"hljs-literal\">None\u003C/span>\u003C/span>) -> \u003Cspan class=\"hljs-type\">Dict\u003C/span>[\u003Cspan class=\"hljs-built_in\">str\u003C/span>, \u003Cspan class=\"hljs-type\">Any\u003C/span>]:\n        \u003Cspan class=\"hljs-string\">"""Generate performance summary report"""\u003C/span>\n        \n        \u003Cspan class=\"hljs-comment\"># Filter metrics\u003C/span>\n        filtered_metrics = self.metrics_history\n        \n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> operation_type:\n            filtered_metrics = [m \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> filtered_metrics \u003Cspan class=\"hljs-keyword\">if\u003C/span> m.operation_type == operation_type]\n        \n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> time_window:\n            cutoff = datetime.now() - time_window\n            filtered_metrics = [m \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> filtered_metrics \u003Cspan class=\"hljs-keyword\">if\u003C/span> m.start_time >= cutoff]\n        \n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> \u003Cspan class=\"hljs-keyword\">not\u003C/span> filtered_metrics:\n            \u003Cspan class=\"hljs-keyword\">return\u003C/span> {\u003Cspan class=\"hljs-string\">'error'\u003C/span>: \u003Cspan class=\"hljs-string\">'No metrics found for criteria'\u003C/span>}\n        \n        \u003Cspan class=\"hljs-comment\"># Calculate statistics\u003C/span>\n        durations = [m.duration_seconds \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> filtered_metrics \u003Cspan class=\"hljs-keyword\">if\u003C/span> m.duration_seconds \u003Cspan class=\"hljs-keyword\">is\u003C/span> \u003Cspan class=\"hljs-keyword\">not\u003C/span> \u003Cspan class=\"hljs-literal\">None\u003C/span>]\n        memory_usage = [m.memory_usage_mb \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> filtered_metrics \u003Cspan class=\"hljs-keyword\">if\u003C/span> m.memory_usage_mb \u003Cspan class=\"hljs-keyword\">is\u003C/span> \u003Cspan class=\"hljs-keyword\">not\u003C/span> \u003Cspan class=\"hljs-literal\">None\u003C/span>]\n        files_processed = [m.files_processed \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> filtered_metrics \u003Cspan class=\"hljs-keyword\">if\u003C/span> m.files_processed \u003Cspan class=\"hljs-keyword\">is\u003C/span> \u003Cspan class=\"hljs-keyword\">not\u003C/span> \u003Cspan class=\"hljs-literal\">None\u003C/span>]\n        \n        summary = {\n            \u003Cspan class=\"hljs-string\">'operation_count'\u003C/span>: \u003Cspan class=\"hljs-built_in\">len\u003C/span>(filtered_metrics),\n            \u003Cspan class=\"hljs-string\">'time_period'\u003C/span>: \u003Cspan class=\"hljs-built_in\">str\u003C/span>(time_window) \u003Cspan class=\"hljs-keyword\">if\u003C/span> time_window \u003Cspan class=\"hljs-keyword\">else\u003C/span> \u003Cspan class=\"hljs-string\">'All time'\u003C/span>,\n            \u003Cspan class=\"hljs-string\">'operation_type'\u003C/span>: operation_type \u003Cspan class=\"hljs-keyword\">or\u003C/span> \u003Cspan class=\"hljs-string\">'All types'\u003C/span>\n        }\n        \n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> durations:\n            summary[\u003Cspan class=\"hljs-string\">'timing'\u003C/span>] = {\n                \u003Cspan class=\"hljs-string\">'avg_duration'\u003C/span>: statistics.mean(durations),\n                \u003Cspan class=\"hljs-string\">'min_duration'\u003C/span>: \u003Cspan class=\"hljs-built_in\">min\u003C/span>(durations),\n                \u003Cspan class=\"hljs-string\">'max_duration'\u003C/span>: \u003Cspan class=\"hljs-built_in\">max\u003C/span>(durations),\n                \u003Cspan class=\"hljs-string\">'median_duration'\u003C/span>: statistics.median(durations),\n                \u003Cspan class=\"hljs-string\">'p95_duration'\u003C/span>: self._percentile(durations, \u003Cspan class=\"hljs-number\">95\u003C/span>),\n                \u003Cspan class=\"hljs-string\">'total_time'\u003C/span>: \u003Cspan class=\"hljs-built_in\">sum\u003C/span>(durations)\n            }\n        \n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> memory_usage:\n            summary[\u003Cspan class=\"hljs-string\">'memory'\u003C/span>] = {\n                \u003Cspan class=\"hljs-string\">'avg_memory_mb'\u003C/span>: statistics.mean(memory_usage),\n                \u003Cspan class=\"hljs-string\">'peak_memory_mb'\u003C/span>: \u003Cspan class=\"hljs-built_in\">max\u003C/span>(memory_usage),\n                \u003Cspan class=\"hljs-string\">'min_memory_mb'\u003C/span>: \u003Cspan class=\"hljs-built_in\">min\u003C/span>(memory_usage)\n            }\n        \n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> files_processed:\n            total_files = \u003Cspan class=\"hljs-built_in\">sum\u003C/span>(files_processed)\n            total_time = \u003Cspan class=\"hljs-built_in\">sum\u003C/span>(durations) \u003Cspan class=\"hljs-keyword\">if\u003C/span> durations \u003Cspan class=\"hljs-keyword\">else\u003C/span> \u003Cspan class=\"hljs-number\">1\u003C/span>\n            \n            summary[\u003Cspan class=\"hljs-string\">'throughput'\u003C/span>] = {\n                \u003Cspan class=\"hljs-string\">'total_files'\u003C/span>: total_files,\n                \u003Cspan class=\"hljs-string\">'avg_files_per_operation'\u003C/span>: statistics.mean(files_processed),\n                \u003Cspan class=\"hljs-string\">'files_per_second'\u003C/span>: total_files / total_time \u003Cspan class=\"hljs-keyword\">if\u003C/span> total_time > \u003Cspan class=\"hljs-number\">0\u003C/span> \u003Cspan class=\"hljs-keyword\">else\u003C/span> \u003Cspan class=\"hljs-number\">0\u003C/span>\n            }\n        \n        \u003Cspan class=\"hljs-comment\"># Cache efficiency\u003C/span>\n        total_hits = \u003Cspan class=\"hljs-built_in\">sum\u003C/span>(m.cache_hits \u003Cspan class=\"hljs-keyword\">or\u003C/span> \u003Cspan class=\"hljs-number\">0\u003C/span> \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> filtered_metrics)\n        total_misses = \u003Cspan class=\"hljs-built_in\">sum\u003C/span>(m.cache_misses \u003Cspan class=\"hljs-keyword\">or\u003C/span> \u003Cspan class=\"hljs-number\">0\u003C/span> \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> filtered_metrics)\n        total_cache_ops = total_hits + total_misses\n        \n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> total_cache_ops > \u003Cspan class=\"hljs-number\">0\u003C/span>:\n            summary[\u003Cspan class=\"hljs-string\">'cache'\u003C/span>] = {\n                \u003Cspan class=\"hljs-string\">'hit_rate'\u003C/span>: (total_hits / total_cache_ops) * \u003Cspan class=\"hljs-number\">100\u003C/span>,\n                \u003Cspan class=\"hljs-string\">'total_hits'\u003C/span>: total_hits,\n                \u003Cspan class=\"hljs-string\">'total_misses'\u003C/span>: total_misses\n            }\n        \n        \u003Cspan class=\"hljs-keyword\">return\u003C/span> summary\n    \n    \u003Cspan class=\"hljs-keyword\">def\u003C/span> \u003Cspan class=\"hljs-title function_\">_percentile\u003C/span>(\u003Cspan class=\"hljs-params\">self, data: \u003Cspan class=\"hljs-type\">List\u003C/span>[\u003Cspan class=\"hljs-built_in\">float\u003C/span>], percentile: \u003Cspan class=\"hljs-built_in\">int\u003C/span>\u003C/span>) -> \u003Cspan class=\"hljs-built_in\">float\u003C/span>:\n        \u003Cspan class=\"hljs-string\">"""Calculate percentile value"""\u003C/span>\n        sorted_data = \u003Cspan class=\"hljs-built_in\">sorted\u003C/span>(data)\n        index = (percentile / \u003Cspan class=\"hljs-number\">100\u003C/span>) * (\u003Cspan class=\"hljs-built_in\">len\u003C/span>(sorted_data) - \u003Cspan class=\"hljs-number\">1\u003C/span>)\n        \n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> index.is_integer():\n            \u003Cspan class=\"hljs-keyword\">return\u003C/span> sorted_data[\u003Cspan class=\"hljs-built_in\">int\u003C/span>(index)]\n        \u003Cspan class=\"hljs-keyword\">else\u003C/span>:\n            lower = sorted_data[\u003Cspan class=\"hljs-built_in\">int\u003C/span>(index)]\n            upper = sorted_data[\u003Cspan class=\"hljs-built_in\">int\u003C/span>(index) + \u003Cspan class=\"hljs-number\">1\u003C/span>]\n            \u003Cspan class=\"hljs-keyword\">return\u003C/span> lower + (upper - lower) * (index - \u003Cspan class=\"hljs-built_in\">int\u003C/span>(index))\n    \n    \u003Cspan class=\"hljs-keyword\">def\u003C/span> \u003Cspan class=\"hljs-title function_\">detect_performance_bottlenecks\u003C/span>(\u003Cspan class=\"hljs-params\">self\u003C/span>) -> \u003Cspan class=\"hljs-type\">List\u003C/span>[\u003Cspan class=\"hljs-type\">Dict\u003C/span>[\u003Cspan class=\"hljs-built_in\">str\u003C/span>, \u003Cspan class=\"hljs-type\">Any\u003C/span>]]:\n        \u003Cspan class=\"hljs-string\">"""Detect performance bottlenecks and anomalies"""\u003C/span>\n        bottlenecks = []\n        \n        \u003Cspan class=\"hljs-comment\"># Analyze recent operations\u003C/span>\n        recent_window = timedelta(minutes=\u003Cspan class=\"hljs-number\">30\u003C/span>)\n        recent_metrics = [\n            m \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> self.metrics_history \n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> m.start_time >= datetime.now() - recent_window\n        ]\n        \n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> \u003Cspan class=\"hljs-built_in\">len\u003C/span>(recent_metrics) < \u003Cspan class=\"hljs-number\">5\u003C/span>:\n            \u003Cspan class=\"hljs-keyword\">return\u003C/span> bottlenecks\n        \n        \u003Cspan class=\"hljs-comment\"># Check for slow operations\u003C/span>\n        durations = [m.duration_seconds \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> recent_metrics \u003Cspan class=\"hljs-keyword\">if\u003C/span> m.duration_seconds]\n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> durations:\n            avg_duration = statistics.mean(durations)\n            threshold = avg_duration * \u003Cspan class=\"hljs-number\">2\u003C/span>\n            \n            slow_ops = [m \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> recent_metrics \u003Cspan class=\"hljs-keyword\">if\u003C/span> m.duration_seconds \u003Cspan class=\"hljs-keyword\">and\u003C/span> m.duration_seconds > threshold]\n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> slow_ops:\n                bottlenecks.append({\n                    \u003Cspan class=\"hljs-string\">'type'\u003C/span>: \u003Cspan class=\"hljs-string\">'slow_operations'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'count'\u003C/span>: \u003Cspan class=\"hljs-built_in\">len\u003C/span>(slow_ops),\n                    \u003Cspan class=\"hljs-string\">'threshold_seconds'\u003C/span>: threshold,\n                    \u003Cspan class=\"hljs-string\">'details'\u003C/span>: [\n                        {\n                            \u003Cspan class=\"hljs-string\">'operation_id'\u003C/span>: op.operation_id,\n                            \u003Cspan class=\"hljs-string\">'duration'\u003C/span>: op.duration_seconds,\n                            \u003Cspan class=\"hljs-string\">'type'\u003C/span>: op.operation_type\n                        }\n                        \u003Cspan class=\"hljs-keyword\">for\u003C/span> op \u003Cspan class=\"hljs-keyword\">in\u003C/span> slow_ops[:\u003Cspan class=\"hljs-number\">5\u003C/span>]  \u003Cspan class=\"hljs-comment\"># Show top 5\u003C/span>\n                    ]\n                })\n        \n        \u003Cspan class=\"hljs-comment\"># Check for memory issues\u003C/span>\n        memory_usage = [m.memory_usage_mb \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> recent_metrics \u003Cspan class=\"hljs-keyword\">if\u003C/span> m.memory_usage_mb]\n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> memory_usage:\n            avg_memory = statistics.mean(memory_usage)\n            high_memory_threshold = avg_memory * \u003Cspan class=\"hljs-number\">1.5\u003C/span>\n            \n            high_memory_ops = [m \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> recent_metrics \u003Cspan class=\"hljs-keyword\">if\u003C/span> m.memory_usage_mb \u003Cspan class=\"hljs-keyword\">and\u003C/span> m.memory_usage_mb > high_memory_threshold]\n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> high_memory_ops:\n                bottlenecks.append({\n                    \u003Cspan class=\"hljs-string\">'type'\u003C/span>: \u003Cspan class=\"hljs-string\">'high_memory_usage'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'count'\u003C/span>: \u003Cspan class=\"hljs-built_in\">len\u003C/span>(high_memory_ops),\n                    \u003Cspan class=\"hljs-string\">'threshold_mb'\u003C/span>: high_memory_threshold,\n                    \u003Cspan class=\"hljs-string\">'peak_usage_mb'\u003C/span>: \u003Cspan class=\"hljs-built_in\">max\u003C/span>(m.memory_usage_mb \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> high_memory_ops)\n                })\n        \n        \u003Cspan class=\"hljs-comment\"># Check cache efficiency\u003C/span>\n        total_hits = \u003Cspan class=\"hljs-built_in\">sum\u003C/span>(m.cache_hits \u003Cspan class=\"hljs-keyword\">or\u003C/span> \u003Cspan class=\"hljs-number\">0\u003C/span> \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> recent_metrics)\n        total_misses = \u003Cspan class=\"hljs-built_in\">sum\u003C/span>(m.cache_misses \u003Cspan class=\"hljs-keyword\">or\u003C/span> \u003Cspan class=\"hljs-number\">0\u003C/span> \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> recent_metrics)\n        total_cache_ops = total_hits + total_misses\n        \n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> total_cache_ops > \u003Cspan class=\"hljs-number\">0\u003C/span>:\n            hit_rate = (total_hits / total_cache_ops) * \u003Cspan class=\"hljs-number\">100\u003C/span>\n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> hit_rate < \u003Cspan class=\"hljs-number\">70\u003C/span>:  \u003Cspan class=\"hljs-comment\"># Less than 70% hit rate\u003C/span>\n                bottlenecks.append({\n                    \u003Cspan class=\"hljs-string\">'type'\u003C/span>: \u003Cspan class=\"hljs-string\">'low_cache_efficiency'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'hit_rate'\u003C/span>: hit_rate,\n                    \u003Cspan class=\"hljs-string\">'total_operations'\u003C/span>: total_cache_ops,\n                    \u003Cspan class=\"hljs-string\">'recommendation'\u003C/span>: \u003Cspan class=\"hljs-string\">'Consider increasing cache size or TTL'\u003C/span>\n                })\n        \n        \u003Cspan class=\"hljs-comment\"># Check system resource usage\u003C/span>\n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> self.system_metrics:\n            recent_system = [\n                m \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> self.system_metrics \n                \u003Cspan class=\"hljs-keyword\">if\u003C/span> m[\u003Cspan class=\"hljs-string\">'timestamp'\u003C/span>] >= datetime.now() - recent_window\n            ]\n            \n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> recent_system:\n                avg_cpu = statistics.mean(m[\u003Cspan class=\"hljs-string\">'cpu_percent'\u003C/span>] \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> recent_system)\n                avg_memory = statistics.mean(m[\u003Cspan class=\"hljs-string\">'memory_percent'\u003C/span>] \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> recent_system)\n                \n                \u003Cspan class=\"hljs-keyword\">if\u003C/span> avg_cpu > \u003Cspan class=\"hljs-number\">80\u003C/span>:\n                    bottlenecks.append({\n                        \u003Cspan class=\"hljs-string\">'type'\u003C/span>: \u003Cspan class=\"hljs-string\">'high_cpu_usage'\u003C/span>,\n                        \u003Cspan class=\"hljs-string\">'avg_cpu_percent'\u003C/span>: avg_cpu,\n                        \u003Cspan class=\"hljs-string\">'recommendation'\u003C/span>: \u003Cspan class=\"hljs-string\">'Consider optimizing algorithms or scaling horizontally'\u003C/span>\n                    })\n                \n                \u003Cspan class=\"hljs-keyword\">if\u003C/span> avg_memory > \u003Cspan class=\"hljs-number\">85\u003C/span>:\n                    bottlenecks.append({\n                        \u003Cspan class=\"hljs-string\">'type'\u003C/span>: \u003Cspan class=\"hljs-string\">'high_memory_pressure'\u003C/span>,\n                        \u003Cspan class=\"hljs-string\">'avg_memory_percent'\u003C/span>: avg_memory,\n                        \u003Cspan class=\"hljs-string\">'recommendation'\u003C/span>: \u003Cspan class=\"hljs-string\">'Consider increasing available memory or optimizing memory usage'\u003C/span>\n                    })\n        \n        \u003Cspan class=\"hljs-keyword\">return\u003C/span> bottlenecks\n    \n    \u003Cspan class=\"hljs-keyword\">def\u003C/span> \u003Cspan class=\"hljs-title function_\">optimize_generation_strategy\u003C/span>(\u003Cspan class=\"hljs-params\">self\u003C/span>) -> \u003Cspan class=\"hljs-type\">Dict\u003C/span>[\u003Cspan class=\"hljs-built_in\">str\u003C/span>, \u003Cspan class=\"hljs-type\">Any\u003C/span>]:\n        \u003Cspan class=\"hljs-string\">"""Generate optimization recommendations based on performance data"""\u003C/span>\n        summary = self.get_performance_summary(time_window=timedelta(hours=\u003Cspan class=\"hljs-number\">24\u003C/span>))\n        bottlenecks = self.detect_performance_bottlenecks()\n        \n        recommendations = []\n        \n        \u003Cspan class=\"hljs-comment\"># Analyze timing performance\u003C/span>\n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> \u003Cspan class=\"hljs-string\">'timing'\u003C/span> \u003Cspan class=\"hljs-keyword\">in\u003C/span> summary:\n            avg_duration = summary[\u003Cspan class=\"hljs-string\">'timing'\u003C/span>][\u003Cspan class=\"hljs-string\">'avg_duration'\u003C/span>]\n            p95_duration = summary[\u003Cspan class=\"hljs-string\">'timing'\u003C/span>][\u003Cspan class=\"hljs-string\">'p95_duration'\u003C/span>]\n            \n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> avg_duration > \u003Cspan class=\"hljs-number\">30\u003C/span>:  \u003Cspan class=\"hljs-comment\"># 30 seconds\u003C/span>\n                recommendations.append({\n                    \u003Cspan class=\"hljs-string\">'category'\u003C/span>: \u003Cspan class=\"hljs-string\">'performance'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'priority'\u003C/span>: \u003Cspan class=\"hljs-string\">'high'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'recommendation'\u003C/span>: \u003Cspan class=\"hljs-string\">'Consider implementing parallel processing for content generation'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'rationale'\u003C/span>: \u003Cspan class=\"hljs-string\">f'Average generation time is \u003Cspan class=\"hljs-subst\">{avg_duration:\u003Cspan class=\"hljs-number\">.1\u003C/span>f}\u003C/span> seconds'\u003C/span>\n                })\n            \n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> p95_duration > avg_duration * \u003Cspan class=\"hljs-number\">3\u003C/span>:\n                recommendations.append({\n                    \u003Cspan class=\"hljs-string\">'category'\u003C/span>: \u003Cspan class=\"hljs-string\">'consistency'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'priority'\u003C/span>: \u003Cspan class=\"hljs-string\">'medium'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'recommendation'\u003C/span>: \u003Cspan class=\"hljs-string\">'Investigate high variance in processing times'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'rationale'\u003C/span>: \u003Cspan class=\"hljs-string\">f'95th percentile is \u003Cspan class=\"hljs-subst\">{p95_duration/avg_duration:\u003Cspan class=\"hljs-number\">.1\u003C/span>f}\u003C/span>x average'\u003C/span>\n                })\n        \n        \u003Cspan class=\"hljs-comment\"># Analyze throughput\u003C/span>\n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> \u003Cspan class=\"hljs-string\">'throughput'\u003C/span> \u003Cspan class=\"hljs-keyword\">in\u003C/span> summary:\n            files_per_second = summary[\u003Cspan class=\"hljs-string\">'throughput'\u003C/span>][\u003Cspan class=\"hljs-string\">'files_per_second'\u003C/span>]\n            \n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> files_per_second < \u003Cspan class=\"hljs-number\">1\u003C/span>:\n                recommendations.append({\n                    \u003Cspan class=\"hljs-string\">'category'\u003C/span>: \u003Cspan class=\"hljs-string\">'throughput'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'priority'\u003C/span>: \u003Cspan class=\"hljs-string\">'high'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'recommendation'\u003C/span>: \u003Cspan class=\"hljs-string\">'Implement batch processing and template caching'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'rationale'\u003C/span>: \u003Cspan class=\"hljs-string\">f'Current throughput is only \u003Cspan class=\"hljs-subst\">{files_per_second:\u003Cspan class=\"hljs-number\">.2\u003C/span>f}\u003C/span> files/second'\u003C/span>\n                })\n        \n        \u003Cspan class=\"hljs-comment\"># Cache optimization\u003C/span>\n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> \u003Cspan class=\"hljs-string\">'cache'\u003C/span> \u003Cspan class=\"hljs-keyword\">in\u003C/span> summary \u003Cspan class=\"hljs-keyword\">and\u003C/span> summary[\u003Cspan class=\"hljs-string\">'cache'\u003C/span>][\u003Cspan class=\"hljs-string\">'hit_rate'\u003C/span>] < \u003Cspan class=\"hljs-number\">80\u003C/span>:\n            recommendations.append({\n                \u003Cspan class=\"hljs-string\">'category'\u003C/span>: \u003Cspan class=\"hljs-string\">'caching'\u003C/span>,\n                \u003Cspan class=\"hljs-string\">'priority'\u003C/span>: \u003Cspan class=\"hljs-string\">'medium'\u003C/span>,\n                \u003Cspan class=\"hljs-string\">'recommendation'\u003C/span>: \u003Cspan class=\"hljs-string\">'Optimize cache configuration and increase cache TTL'\u003C/span>,\n                \u003Cspan class=\"hljs-string\">'rationale'\u003C/span>: \u003Cspan class=\"hljs-string\">f"Cache hit rate is only \u003Cspan class=\"hljs-subst\">{summary[\u003Cspan class=\"hljs-string\">'cache'\u003C/span>][\u003Cspan class=\"hljs-string\">'hit_rate'\u003C/span>]:\u003Cspan class=\"hljs-number\">.1\u003C/span>f}\u003C/span>%"\u003C/span>\n            })\n        \n        \u003Cspan class=\"hljs-comment\"># Memory optimization\u003C/span>\n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> \u003Cspan class=\"hljs-string\">'memory'\u003C/span> \u003Cspan class=\"hljs-keyword\">in\u003C/span> summary:\n            peak_memory = summary[\u003Cspan class=\"hljs-string\">'memory'\u003C/span>][\u003Cspan class=\"hljs-string\">'peak_memory_mb'\u003C/span>]\n            \n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> peak_memory > \u003Cspan class=\"hljs-number\">1000\u003C/span>:  \u003Cspan class=\"hljs-comment\"># 1GB\u003C/span>\n                recommendations.append({\n                    \u003Cspan class=\"hljs-string\">'category'\u003C/span>: \u003Cspan class=\"hljs-string\">'memory'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'priority'\u003C/span>: \u003Cspan class=\"hljs-string\">'medium'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'recommendation'\u003C/span>: \u003Cspan class=\"hljs-string\">'Implement streaming processing for large datasets'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'rationale'\u003C/span>: \u003Cspan class=\"hljs-string\">f'Peak memory usage is \u003Cspan class=\"hljs-subst\">{peak_memory:\u003Cspan class=\"hljs-number\">.0\u003C/span>f}\u003C/span>MB'\u003C/span>\n                })\n        \n        \u003Cspan class=\"hljs-comment\"># Bottleneck-specific recommendations\u003C/span>\n        \u003Cspan class=\"hljs-keyword\">for\u003C/span> bottleneck \u003Cspan class=\"hljs-keyword\">in\u003C/span> bottlenecks:\n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> bottleneck[\u003Cspan class=\"hljs-string\">'type'\u003C/span>] == \u003Cspan class=\"hljs-string\">'slow_operations'\u003C/span>:\n                recommendations.append({\n                    \u003Cspan class=\"hljs-string\">'category'\u003C/span>: \u003Cspan class=\"hljs-string\">'performance'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'priority'\u003C/span>: \u003Cspan class=\"hljs-string\">'high'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'recommendation'\u003C/span>: \u003Cspan class=\"hljs-string\">'Profile and optimize slow operations'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'rationale'\u003C/span>: \u003Cspan class=\"hljs-string\">f"\u003Cspan class=\"hljs-subst\">{bottleneck[\u003Cspan class=\"hljs-string\">'count'\u003C/span>]}\u003C/span> operations exceeded normal duration"\u003C/span>\n                })\n            \u003Cspan class=\"hljs-keyword\">elif\u003C/span> bottleneck[\u003Cspan class=\"hljs-string\">'type'\u003C/span>] == \u003Cspan class=\"hljs-string\">'low_cache_efficiency'\u003C/span>:\n                recommendations.append({\n                    \u003Cspan class=\"hljs-string\">'category'\u003C/span>: \u003Cspan class=\"hljs-string\">'caching'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'priority'\u003C/span>: \u003Cspan class=\"hljs-string\">'medium'\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'recommendation'\u003C/span>: bottleneck[\u003Cspan class=\"hljs-string\">'recommendation'\u003C/span>],\n                    \u003Cspan class=\"hljs-string\">'rationale'\u003C/span>: \u003Cspan class=\"hljs-string\">f"Cache hit rate is \u003Cspan class=\"hljs-subst\">{bottleneck[\u003Cspan class=\"hljs-string\">'hit_rate'\u003C/span>]:\u003Cspan class=\"hljs-number\">.1\u003C/span>f}\u003C/span>%"\u003C/span>\n                })\n        \n        \u003Cspan class=\"hljs-keyword\">return\u003C/span> {\n            \u003Cspan class=\"hljs-string\">'analysis_timestamp'\u003C/span>: datetime.now().isoformat(),\n            \u003Cspan class=\"hljs-string\">'performance_summary'\u003C/span>: summary,\n            \u003Cspan class=\"hljs-string\">'bottlenecks'\u003C/span>: bottlenecks,\n            \u003Cspan class=\"hljs-string\">'recommendations'\u003C/span>: recommendations,\n            \u003Cspan class=\"hljs-string\">'optimization_score'\u003C/span>: self._calculate_optimization_score(summary, bottlenecks)\n        }\n    \n    \u003Cspan class=\"hljs-keyword\">def\u003C/span> \u003Cspan class=\"hljs-title function_\">_calculate_optimization_score\u003C/span>(\u003Cspan class=\"hljs-params\">self, summary: \u003Cspan class=\"hljs-type\">Dict\u003C/span>[\u003Cspan class=\"hljs-built_in\">str\u003C/span>, \u003Cspan class=\"hljs-type\">Any\u003C/span>], bottlenecks: \u003Cspan class=\"hljs-type\">List\u003C/span>[\u003Cspan class=\"hljs-type\">Dict\u003C/span>[\u003Cspan class=\"hljs-built_in\">str\u003C/span>, \u003Cspan class=\"hljs-type\">Any\u003C/span>]]\u003C/span>) -> \u003Cspan class=\"hljs-built_in\">int\u003C/span>:\n        \u003Cspan class=\"hljs-string\">"""Calculate optimization score (0-100)"""\u003C/span>\n        score = \u003Cspan class=\"hljs-number\">100\u003C/span>\n        \n        \u003Cspan class=\"hljs-comment\"># Deduct points for performance issues\u003C/span>\n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> \u003Cspan class=\"hljs-string\">'timing'\u003C/span> \u003Cspan class=\"hljs-keyword\">in\u003C/span> summary:\n            avg_duration = summary[\u003Cspan class=\"hljs-string\">'timing'\u003C/span>][\u003Cspan class=\"hljs-string\">'avg_duration'\u003C/span>]\n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> avg_duration > \u003Cspan class=\"hljs-number\">60\u003C/span>:\n                score -= \u003Cspan class=\"hljs-number\">20\u003C/span>\n            \u003Cspan class=\"hljs-keyword\">elif\u003C/span> avg_duration > \u003Cspan class=\"hljs-number\">30\u003C/span>:\n                score -= \u003Cspan class=\"hljs-number\">10\u003C/span>\n        \n        \u003Cspan class=\"hljs-keyword\">if\u003C/span> \u003Cspan class=\"hljs-string\">'cache'\u003C/span> \u003Cspan class=\"hljs-keyword\">in\u003C/span> summary:\n            hit_rate = summary[\u003Cspan class=\"hljs-string\">'cache'\u003C/span>][\u003Cspan class=\"hljs-string\">'hit_rate'\u003C/span>]\n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> hit_rate < \u003Cspan class=\"hljs-number\">50\u003C/span>:\n                score -= \u003Cspan class=\"hljs-number\">20\u003C/span>\n            \u003Cspan class=\"hljs-keyword\">elif\u003C/span> hit_rate < \u003Cspan class=\"hljs-number\">80\u003C/span>:\n                score -= \u003Cspan class=\"hljs-number\">10\u003C/span>\n        \n        \u003Cspan class=\"hljs-comment\"># Deduct points for bottlenecks\u003C/span>\n        \u003Cspan class=\"hljs-keyword\">for\u003C/span> bottleneck \u003Cspan class=\"hljs-keyword\">in\u003C/span> bottlenecks:\n            \u003Cspan class=\"hljs-keyword\">if\u003C/span> bottleneck[\u003Cspan class=\"hljs-string\">'type'\u003C/span>] \u003Cspan class=\"hljs-keyword\">in\u003C/span> [\u003Cspan class=\"hljs-string\">'slow_operations'\u003C/span>, \u003Cspan class=\"hljs-string\">'high_memory_usage'\u003C/span>]:\n                score -= \u003Cspan class=\"hljs-number\">15\u003C/span>\n            \u003Cspan class=\"hljs-keyword\">else\u003C/span>:\n                score -= \u003Cspan class=\"hljs-number\">5\u003C/span>\n        \n        \u003Cspan class=\"hljs-keyword\">return\u003C/span> \u003Cspan class=\"hljs-built_in\">max\u003C/span>(\u003Cspan class=\"hljs-number\">0\u003C/span>, score)\n    \n    \u003Cspan class=\"hljs-keyword\">def\u003C/span> \u003Cspan class=\"hljs-title function_\">export_metrics\u003C/span>(\u003Cspan class=\"hljs-params\">self, filename: \u003Cspan class=\"hljs-built_in\">str\u003C/span>\u003C/span>):\n        \u003Cspan class=\"hljs-string\">"""Export metrics to JSON file"""\u003C/span>\n        export_data = {\n            \u003Cspan class=\"hljs-string\">'export_timestamp'\u003C/span>: datetime.now().isoformat(),\n            \u003Cspan class=\"hljs-string\">'metrics_history'\u003C/span>: [\n                {\n                    \u003Cspan class=\"hljs-string\">'operation_id'\u003C/span>: m.operation_id,\n                    \u003Cspan class=\"hljs-string\">'operation_type'\u003C/span>: m.operation_type,\n                    \u003Cspan class=\"hljs-string\">'start_time'\u003C/span>: m.start_time.isoformat(),\n                    \u003Cspan class=\"hljs-string\">'end_time'\u003C/span>: m.end_time.isoformat() \u003Cspan class=\"hljs-keyword\">if\u003C/span> m.end_time \u003Cspan class=\"hljs-keyword\">else\u003C/span> \u003Cspan class=\"hljs-literal\">None\u003C/span>,\n                    \u003Cspan class=\"hljs-string\">'duration_seconds'\u003C/span>: m.duration_seconds,\n                    \u003Cspan class=\"hljs-string\">'memory_usage_mb'\u003C/span>: m.memory_usage_mb,\n                    \u003Cspan class=\"hljs-string\">'cpu_usage_percent'\u003C/span>: m.cpu_usage_percent,\n                    \u003Cspan class=\"hljs-string\">'files_processed'\u003C/span>: m.files_processed,\n                    \u003Cspan class=\"hljs-string\">'output_size_kb'\u003C/span>: m.output_size_kb,\n                    \u003Cspan class=\"hljs-string\">'cache_hits'\u003C/span>: m.cache_hits,\n                    \u003Cspan class=\"hljs-string\">'cache_misses'\u003C/span>: m.cache_misses\n                }\n                \u003Cspan class=\"hljs-keyword\">for\u003C/span> m \u003Cspan class=\"hljs-keyword\">in\u003C/span> self.metrics_history\n            ],\n            \u003Cspan class=\"hljs-string\">'system_metrics'\u003C/span>: self.system_metrics[-\u003Cspan class=\"hljs-number\">100\u003C/span>:],  \u003Cspan class=\"hljs-comment\"># Last 100 system samples\u003C/span>\n            \u003Cspan class=\"hljs-string\">'summary'\u003C/span>: self.get_performance_summary(),\n            \u003Cspan class=\"hljs-string\">'optimization_analysis'\u003C/span>: self.optimize_generation_strategy()\n        }\n        \n        \u003Cspan class=\"hljs-keyword\">with\u003C/span> \u003Cspan class=\"hljs-built_in\">open\u003C/span>(filename, \u003Cspan class=\"hljs-string\">'w'\u003C/span>) \u003Cspan class=\"hljs-keyword\">as\u003C/span> f:\n            json.dump(export_data, f, indent=\u003Cspan class=\"hljs-number\">2\u003C/span>, default=\u003Cspan class=\"hljs-built_in\">str\u003C/span>)\n\n\u003Cspan class=\"hljs-comment\"># Global performance monitor instance\u003C/span>\nperformance_monitor = ContentGenerationPerformanceMonitor()\n\u003C/code>\u003C/pre>\u003Ch2 id=\"conclusion\">Conclusion\u003C/h2>\n\u003Cp>Advanced Markdown dynamic content generation and automation workflows represent transformative approaches to scalable technical documentation that enable systematic creation of data-driven content while maintaining consistency, accuracy, and efficiency across large documentation ecosystems. By implementing comprehensive automation strategies, template engines, and intelligent content pipelines, technical writers can build documentation systems that adapt dynamically to changing requirements while reducing manual effort and ensuring quality standards.\u003C/p>\n\u003Cp>The key to successful content automation lies in understanding the relationship between data sources, template design, and output requirements, enabling the creation of flexible systems that can accommodate diverse content types while maintaining performance and reliability at scale. Whether you're generating API documentation, metrics reports, or feature guides, the automation techniques and infrastructure patterns covered in this guide provide the foundation for building robust content generation systems.\u003C/p>\n\u003Cp>Remember to implement automation incrementally, starting with simple template-based generation before adding sophisticated data integration and processing capabilities, and ensure that generated content maintains the same quality standards as manually created content through comprehensive validation and testing processes. With proper implementation of dynamic content generation workflows, your documentation system can achieve unprecedented scalability while maintaining the accuracy and consistency essential for effective technical communication.\u003C/p>\n",[15,19,22,25,28,31,34],{"id":16,"text":17,"level":18},"why-master-dynamic-content-generation","Why Master Dynamic Content Generation?",2,{"id":20,"text":21,"level":18},"foundation-automation-principles","Foundation Automation Principles",{"id":23,"text":24,"level":18},"integration-with-cicd-systems","Integration with CI/CD Systems",{"id":26,"text":27,"level":18},"integration-with-documentation-systems","Integration with Documentation Systems",{"id":29,"text":30,"level":18},"advanced-automation-patterns","Advanced Automation Patterns",{"id":32,"text":33,"level":18},"performance-optimization-and-scaling","Performance Optimization and Scaling",{"id":35,"text":36,"level":18},"conclusion","Conclusion",[],[39,47,55],{"slug":40,"title":41,"description":42,"excerpt":43,"date":44,"category":9,"author":10,"keywords":45,"readingMinutes":46},"markdown-dynamic-content-generation-complete-guide","Markdown Dynamic Content Generation: Complete Guide for Automated Documentation and Data-Driven Content Creation","Master dynamic Markdown content generation with templating systems, data binding, automated documentation generation, and programmatic content creation. Learn to build scalable, maintainable documentation workflows that generate content from data sources automatically.","Dynamic Markdown content generation transforms static documentation workflows into powerful, data-driven systems that automatically create, update, and maintain content based on external data…","2025-10-13","markdown dynamic content, markdown templates, automated documentation generation, data-driven markdown, markdown templating systems, programmatic content creation, dynamic documentation workflows, markdown automation",33,{"slug":48,"title":49,"description":50,"excerpt":51,"date":52,"category":9,"author":10,"keywords":53,"readingMinutes":54},"markdown-automation-workflows-complete-guide","Markdown Automation Workflows: Complete Guide for CI/CD, Publishing, and Documentation Generation","Master automated Markdown workflows for continuous documentation, publishing pipelines, and content generation. Learn GitHub Actions, automated testing, deployment strategies, and integration with modern development tools.","Automated Markdown workflows enable seamless integration of documentation and content creation into modern development processes, providing continuous integration capabilities for technical…","2025-09-25","markdown automation, markdown ci cd, automated documentation, markdown workflows, markdown publishing automation, documentation generation, markdown testing, automated content publishing",29,{"slug":56,"title":57,"description":58,"excerpt":59,"date":60,"category":9,"author":10,"keywords":61,"readingMinutes":62},"markdown-diagram-workflows-integration-complete-guide","Markdown Diagram Workflows and Integration: Complete Guide for Visual Documentation and Automated Diagram Generation","Master advanced Markdown diagram workflows with Mermaid, PlantUML, and D2. Learn automated diagram generation, version control integration, CI/CD pipelines, and cross-platform compatibility for professional visual documentation.","Advanced Markdown diagram workflows and integration techniques enable sophisticated visual documentation that combines the simplicity of text-based markup with powerful automated diagram…","2025-11-03","markdown diagrams, mermaid diagrams, plantuml markdown, d2 diagrams, diagram automation, visual documentation, markdown workflows, diagram version control, automated diagrams",16,1792184946037]