{"id":7340,"date":"2024-06-07T08:55:00","date_gmt":"2024-06-07T08:55:00","guid":{"rendered":"https:\/\/megagon.ai\/publications\/cmdbenchmark-for-compound-ai-systems\/"},"modified":"2025-10-23T20:50:41","modified_gmt":"2025-10-23T20:50:41","slug":"cmdbenchmark-for-compound-ai-systems","status":"publish","type":"publications","link":"https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/","title":{"rendered":"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems"},"template":"","publications-tags":[193],"conference-year":[67],"conference":[180],"class_list":["post-7340","publications","type-publications","status-publish","has-post-thumbnail","hentry","publications-tags-compound-ai","conference-year-67","conference-guide-ai-sigmod"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems - Megagon<\/title>\n<meta name=\"description\" content=\"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems Megagon Labs paper. Presented at Guide AI - SIGMOD 2024.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/\" \/>\n<meta property=\"og:locale\" content=\"ja_JP\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems\" \/>\n<meta property=\"og:description\" content=\"We adapt existing datasets and benchmarks in open-domain -- from question answering and complex reasoning tasks to natural language querying over structured data -- to evaluate coarse- and fine-grained data discovery and task execution performance.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/\" \/>\n<meta property=\"og:site_name\" content=\"Megagon\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/megagonlabs\/\" \/>\n<meta property=\"article:modified_time\" content=\"2025-10-23T20:50:41+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:title\" content=\"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems\" \/>\n<meta name=\"twitter:description\" content=\"We adapt existing datasets and benchmarks in open-domain -- from question answering and complex reasoning tasks to natural language querying over structured data -- to evaluate coarse- and fine-grained data discovery and task execution performance.\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/publications\\\/cmdbenchmark-for-compound-ai-systems\\\/\",\"url\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/publications\\\/cmdbenchmark-for-compound-ai-systems\\\/\",\"name\":\"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems - Megagon\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/publications\\\/cmdbenchmark-for-compound-ai-systems\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/publications\\\/cmdbenchmark-for-compound-ai-systems\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/megagon.ai\\\/wp-content\\\/uploads\\\/2025\\\/02\\\/Blue-White-Modern-Business-Technology-Twitter-Post.png\",\"datePublished\":\"2024-06-07T08:55:00+00:00\",\"dateModified\":\"2025-10-23T20:50:41+00:00\",\"description\":\"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems Megagon Labs paper. Presented at Guide AI - SIGMOD 2024.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/publications\\\/cmdbenchmark-for-compound-ai-systems\\\/#breadcrumb\"},\"inLanguage\":\"ja-JP\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/megagon.ai\\\/jp\\\/publications\\\/cmdbenchmark-for-compound-ai-systems\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"ja-JP\",\"@id\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/publications\\\/cmdbenchmark-for-compound-ai-systems\\\/#primaryimage\",\"url\":\"https:\\\/\\\/megagon.ai\\\/wp-content\\\/uploads\\\/2025\\\/02\\\/Blue-White-Modern-Business-Technology-Twitter-Post.png\",\"contentUrl\":\"https:\\\/\\\/megagon.ai\\\/wp-content\\\/uploads\\\/2025\\\/02\\\/Blue-White-Modern-Business-Technology-Twitter-Post.png\",\"width\":1600,\"height\":900,\"caption\":\"CMD BENCH: A benchmark modeling the complexity of enterprise data platforms.\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/publications\\\/cmdbenchmark-for-compound-ai-systems\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Publications\",\"item\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/publications\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/#website\",\"url\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/\",\"name\":\"Megagon Labs\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ja-JP\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/megagon.ai\\\/jp\\\/#organization\",\"name\":\"Megagon Labs\",\"url\":\"https:\\\/\\\/megagon.ai\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https:\\\/\\\/megagon.ai\\\/wp-content\\\/uploads\\\/2025\\\/02\\\/Logo-Megagon-Labs.webp\",\"caption\":\"Megagon Labs\"},\"image\":{\"url\":\"https:\\\/\\\/megagon.ai\\\/wp-content\\\/uploads\\\/2025\\\/02\\\/Logo-Megagon-Labs.webp\"},\"description\":\"Megagon Labs is an AI research organization conducting research in compound AI systems, large language models, data-AI symbiosis, and human-centered AI. Megagon Labs shares its findings with the broader community through open-source tools, datasets, publications, workshops, and an invited speaker series.\",\"sameAs\":[\"https:\\\/\\\/github.com\\\/megagonlabs\",\"https:\\\/\\\/twitter.com\\\/megagonlabs\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/megagon-labs\\\/\",\"https:\\\/\\\/www.facebook.com\\\/megagonlabs\\\/\"],\"address\":{\"@type\":\"PostalAddress\",\"streetAddress\":\"444 Castro Street\",\"addressLocality\":\"Mountain View\",\"addressRegion\":\"CA\",\"postalCode\":\"94041\",\"addressCountry\":\"US\"},\"contactPoint\":{\"@type\":\"ContactPoint\",\"email\":\"contactus@megagon.ai\",\"contactType\":\"general inquiries\"},\"parentOrganization\":{\"@type\":\"Organization\",\"name\":\"Recruit Holdings\",\"url\":\"https:\\\/\\\/recruit-holdings.com\\\/en\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems - Megagon","description":"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems Megagon Labs paper. Presented at Guide AI - SIGMOD 2024.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/","og_locale":"ja_JP","og_type":"article","og_title":"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems","og_description":"We adapt existing datasets and benchmarks in open-domain -- from question answering and complex reasoning tasks to natural language querying over structured data -- to evaluate coarse- and fine-grained data discovery and task execution performance.","og_url":"https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/","og_site_name":"Megagon","article_publisher":"https:\/\/www.facebook.com\/megagonlabs\/","article_modified_time":"2025-10-23T20:50:41+00:00","twitter_card":"summary_large_image","twitter_title":"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems","twitter_description":"We adapt existing datasets and benchmarks in open-domain -- from question answering and complex reasoning tasks to natural language querying over structured data -- to evaluate coarse- and fine-grained data discovery and task execution performance.","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/","url":"https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/","name":"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems - Megagon","isPartOf":{"@id":"https:\/\/megagon.ai\/jp\/#website"},"primaryImageOfPage":{"@id":"https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/#primaryimage"},"image":{"@id":"https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/#primaryimage"},"thumbnailUrl":"https:\/\/megagon.ai\/wp-content\/uploads\/2025\/02\/Blue-White-Modern-Business-Technology-Twitter-Post.png","datePublished":"2024-06-07T08:55:00+00:00","dateModified":"2025-10-23T20:50:41+00:00","description":"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems Megagon Labs paper. Presented at Guide AI - SIGMOD 2024.","breadcrumb":{"@id":"https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/#breadcrumb"},"inLanguage":"ja-JP","potentialAction":[{"@type":"ReadAction","target":["https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/"]}]},{"@type":"ImageObject","inLanguage":"ja-JP","@id":"https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/#primaryimage","url":"https:\/\/megagon.ai\/wp-content\/uploads\/2025\/02\/Blue-White-Modern-Business-Technology-Twitter-Post.png","contentUrl":"https:\/\/megagon.ai\/wp-content\/uploads\/2025\/02\/Blue-White-Modern-Business-Technology-Twitter-Post.png","width":1600,"height":900,"caption":"CMD BENCH: A benchmark modeling the complexity of enterprise data platforms."},{"@type":"BreadcrumbList","@id":"https:\/\/megagon.ai\/jp\/publications\/cmdbenchmark-for-compound-ai-systems\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/megagon.ai\/jp\/"},{"@type":"ListItem","position":2,"name":"Publications","item":"https:\/\/megagon.ai\/jp\/publications\/"},{"@type":"ListItem","position":3,"name":"CMDBench: A Benchmark for Coarse-to-fine Multimodal Data Discovery in Compound AI Systems"}]},{"@type":"WebSite","@id":"https:\/\/megagon.ai\/jp\/#website","url":"https:\/\/megagon.ai\/jp\/","name":"Megagon Labs","description":"","publisher":{"@id":"https:\/\/megagon.ai\/jp\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/megagon.ai\/jp\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ja-JP"},{"@type":"Organization","@id":"https:\/\/megagon.ai\/jp\/#organization","name":"Megagon Labs","url":"https:\/\/megagon.ai\/","logo":{"@type":"ImageObject","url":"https:\/\/megagon.ai\/wp-content\/uploads\/2025\/02\/Logo-Megagon-Labs.webp","caption":"Megagon Labs"},"image":{"url":"https:\/\/megagon.ai\/wp-content\/uploads\/2025\/02\/Logo-Megagon-Labs.webp"},"description":"Megagon Labs is an AI research organization conducting research in compound AI systems, large language models, data-AI symbiosis, and human-centered AI. Megagon Labs shares its findings with the broader community through open-source tools, datasets, publications, workshops, and an invited speaker series.","sameAs":["https:\/\/github.com\/megagonlabs","https:\/\/twitter.com\/megagonlabs","https:\/\/www.linkedin.com\/company\/megagon-labs\/","https:\/\/www.facebook.com\/megagonlabs\/"],"address":{"@type":"PostalAddress","streetAddress":"444 Castro Street","addressLocality":"Mountain View","addressRegion":"CA","postalCode":"94041","addressCountry":"US"},"contactPoint":{"@type":"ContactPoint","email":"contactus@megagon.ai","contactType":"general inquiries"},"parentOrganization":{"@type":"Organization","name":"Recruit Holdings","url":"https:\/\/recruit-holdings.com\/en\/"}}]}},"_links":{"self":[{"href":"https:\/\/megagon.ai\/jp\/wp-json\/wp\/v2\/publications\/7340","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/megagon.ai\/jp\/wp-json\/wp\/v2\/publications"}],"about":[{"href":"https:\/\/megagon.ai\/jp\/wp-json\/wp\/v2\/types\/publications"}],"version-history":[{"count":1,"href":"https:\/\/megagon.ai\/jp\/wp-json\/wp\/v2\/publications\/7340\/revisions"}],"predecessor-version":[{"id":14395,"href":"https:\/\/megagon.ai\/jp\/wp-json\/wp\/v2\/publications\/7340\/revisions\/14395"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/megagon.ai\/jp\/wp-json\/wp\/v2\/media\/7341"}],"wp:attachment":[{"href":"https:\/\/megagon.ai\/jp\/wp-json\/wp\/v2\/media?parent=7340"}],"wp:term":[{"taxonomy":"publications-tags","embeddable":true,"href":"https:\/\/megagon.ai\/jp\/wp-json\/wp\/v2\/publications-tags?post=7340"},{"taxonomy":"conference-year","embeddable":true,"href":"https:\/\/megagon.ai\/jp\/wp-json\/wp\/v2\/conference-year?post=7340"},{"taxonomy":"conference","embeddable":true,"href":"https:\/\/megagon.ai\/jp\/wp-json\/wp\/v2\/conference?post=7340"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}