Docker & Portainer

Turn Container Logs into queryable tables.

Every container the daemon runs writes a log. Query all of them at once — carrying the container's real name, its image and its Compose service, instead of a directory full of 64-character ids.

Use the results in Excel, Google Sheets, AI tools and your apps through the Database REST API.

Every connector type is on Free. Start with 2 data sources and 1 agent.

Connect Container Logs freeConnection details
Container LogsFiles read where they live
Inside your networkQuery Streams Network Agent

Read, organize and query on your machine.

Read-only queries · outbound connection
ExcelGoogle SheetsYour AIYour apps

Source files stay in place. Your tools receive query results.

Source → agent → allowed results

Query Streams turns Docker's json-file logs, the docker logs exports you save by hand, and Portainer's own server log into live SQL tables readable from Microsoft Excel, Google Sheets, AI assistants over MCP, the REST API and Power BI. The hard part it solves is identity: on disk these files are named by container id, so the agent resolves each one back to the container's real name, image and Compose service before the rows land. Nothing is uploaded - the agent reads the files on the host and only query results leave your network.

  • Real container names, not ids
  • Image and Compose service per row
  • stdout and stderr stay separate

Connect Container Logs with the right settings.

Keep the connection on the Network Agent. These are the details specific to Container Logs.

Folder
The daemon's log directory on the container host, or a folder holding exported logs
Sources
Docker json-file logs, saved docker logs exports, and Portainer's server log
Identity
Container name, image and Compose service resolved from the daemon, so rows are not keyed by a 64-character id
Rotation
Followed as the daemon rotates, plus the ten named rotation patterns for exported files
Unmatched lines
Kept verbatim in events_raw_lines, never dropped
Credentials
None - the agent reads the files on the host in place
SQL dialect
DuckDB - standard SQL, nothing container-specific to learn

The agent reads the log files the daemon has already written. It does not attach to containers, does not hold the daemon socket open for streaming, and never writes to or rotates those files itself - so a connector left running cannot change how the host behaves.

Built for how Container Logs works.

The hard part is identity, not parsing

Docker's json-file driver writes each container's log into a directory named after its 64-character id. Parsing the JSON is trivial; knowing which container it was is not, and it is the thing that makes the table usable.

  • Each row carries the container's real name, its image and its Compose service, resolved from the daemon rather than guessed from the path.
  • That means "errors in the api service this morning" is a WHERE clause, not a lookup table you maintain by hand.
  • The stream is kept too, so stdout and stderr stay distinguishable instead of being merged into one column of text.
  • A container that has been removed keeps the rows it already produced - the history does not disappear with the container.
-- Which services are noisiest right now
SELECT   service, stream, COUNT(*) AS lines
FROM     fileset.events
WHERE    stream = 'stderr'
GROUP BY service, stream
ORDER BY lines DESC

Three things, not one

The daemon's json-file logs are the main event, but they are not the only container log worth querying: a docker logs export saved by hand has a different shape, and Portainer writes a server log of its own. All three have profiles, so you choose what you are reading rather than hoping one parser copes with all of it.

The formats and behavior, in detail.

Open the reference you need for Container Logs.

One folder of container logs becomes one table.
The big idea

One folder of container logs becomes one table.

Query Streams reads the files the daemon already writes, pins the format, and gives you typed columns with the container named beside every row. No sidecar, no shipping pipeline, no index to maintain — and a line the parser cannot read is kept as a row with a reason rather than silently dropped.

Log folderon the host
a3f9c1…-json.logDocker
Dockershopfront-stack_api.logexport
portainer.logPortainer

Rotations, compressed archives and re-read files — handled

noisy-containers.sqlSQL
SELECT app,
       compose_service,
       count(*) AS lines
FROM   fileset.events
WHERE  stream = 'stderr'
GROUP BY app, compose_service
ORDER BY lines DESC
Resultstderr by container
appservicelines
shopfront-apiapi1,284
shopfront-workerworker417
shopfront-webweb62
shopfront-cachecache3

Join it to your application database in the same query

Files the daemon already writes, now a table you can report on

Two tools, three formats, no sidecar
Sources

Two tools, three formats, no sidecar

Each card names the exact files Query Streams reads and where they live. You do not change your logging driver to make this work — json-file is the Docker default, and the default is the supported one.

Supported container logs

3 formats · 2 tools
Docker logo - Query Streams reads Docker json-file container logs and docker logs exports Dockerjson-file logging driver
  • One {"log","stream","time"} envelope per line, written by the daemon
  • The container name comes from Docker’s own config.v2.json beside the log
  • Image and Compose project/service ride along as columns
  • Usually /var/lib/docker/containers
  • Appends; numbered rotations and .gz followed
Docker logo - Query Streams reads docker logs --timestamps export files Dockerdocker logs export
  • Files you saved with docker logs --timestamps
  • An RFC 3339 nanosecond stamp, then the container’s own line
  • Container banner lines are skipped by declaration, not by luck
  • Any folder you keep the exports in
  • One file per service; the name comes from the file stem
Portainer logo - Query Streams reads Portainer server zerolog console logs PortainerServer log, zerolog console
  • portainer.log or the container’s own output
  • ANSI colour codes stripped before parsing, so a level is a level
  • The key=value tail becomes columns; the remainder lands in data
  • The embedded tunnel server’s differently shaped lines are read too
  • Minute-resolution stamps, read as UTC
More comingthese are the start, not the ceiling

New container log formats are added with each Agent release. Because Agents keep themselves up to date, a format added later shows up in your connector without anyone visiting a server.

All three parsers were built and tested against real container logs — json-file logs, large docker logs exports off a Swarm host, and Portainer CE 2.39.1’s server log. One thing to know up front: on Docker Desktop for Windows and Mac the json-file logs live inside its virtual machine, where an Agent running as a service cannot reach them. Export them with docker logs --timestamps and use the second profile.
Containers in every region, logs read where they sit
Anywhere it runs

Containers in every region, logs read where they sit

Containers rarely all run in one place. A host in the office, a Swarm cluster in one cloud region, a single box in another country. Each location runs a Network Agent that reads the log files where they already are and dials out to Query Streams. As far as your firewall is concerned that is an ordinary outbound connection — there is nothing to open and no VPN to build.

Head office A Docker host on local disk, json-file logs under the daemon’s folder dials out
eu-west A Swarm cluster, one docker logs export per service dials out
us-east Portainer managing a handful of hosts, its own server log alongside dials out

Three outbound connections, one place to query them — no inbound port, no VPN, no firewall change

One Agent per location

An Agent covers a whole site rather than a single folder: each log format you want there becomes its own File Set connector on that same Agent. One Agent per location is the normal shape. The Free plan runs a single Agent and paid plans run more.

1 site = 1 Agent = many connectors

Nothing to open

The Agent makes one encrypted outbound connection, and the request out and the rows back both travel over it. No inbound port, no VPN, no firewall change to go and ask for — and credentials stay inside your network.

one connection, both directions

One query across sites

Every source in a federated query names its own Agent, so a single statement can read a log in one country and a log in another and hand back one result — or join either of them to a database. Included on Business and Enterprise.

2 countries → 1 result set

They keep themselves current

Agents update themselves, so a container log format added in a later release reaches every site without anyone logging into a server to install it. That is what makes the wall above a starting point rather than a fixed list.

new formats arrive with the Agent

Names, clocks and colour codes
The hard part

Names, clocks and colour codes

A container log is not a text file with a date on it. Three things routinely go wrong when you try to treat one as data, and each is handled here by declaration rather than by guesswork.

Names, not hex ids

Docker names each log folder after the container id, which makes a grouped query unreadable. Query Streams reads the record Docker writes beside the log and takes the real name, keeping the id as its own column.

a3f9c1…-json.log → app = shopfront-api

The daemon’s clock

A container that prints its own timestamp in epoch seconds lands in 1970 if you believe it. The event time is the daemon’s capture time; the container’s own stamp is kept as a column instead of being trusted.

envelope time → event_time

Colour codes in the file

Console loggers write terminal escape sequences straight into the log. They are stripped before parsing, so a level reads as a level rather than as a level wrapped in escape characters — and the original bytes are left untouched.

escape codes removed → level = INF

Nothing counted twice

Each file is fingerprinted by its first 4 KB, so a rotated file is recognised as the same file rather than read again from the top. A rolling duplicate window catches the overlap when a writer replays its tail.

4 KB prefix hash · 20,000-record window

The columns you actually get
The schema

The columns you actually get

Not a blob of text with a timestamp beside it. Each format is parsed into typed columns you can filter, group and aggregate on directly — and where a container logs structured JSON, its own fields are unwrapped into columns too.

Docker logo
json-file container logDocker daemon
streamlevelloggermessagehostservicedata
Docker logo
docker logs exportDocker CLI
message
Portainer logo
zerolog console logPortainer
levelcallermessageerrordata

Every row also carries event_time, the raw stamp it was read from, and the file it came from. Each format adds its identity column — app for json-file and Portainer, container for an export — and json-file rows additionally carry image, compose_project, compose_service and container_id. The single message column on an export is deliberate: the container’s own line is kept whole rather than guessed at.

What this deliberately does not do

What this deliberately does not do

It is not a live tail. Query Streams reads the files as they are on disk when you run the query, so a restart loop shows up as rows rather than as a stream you watch. If you need sub-second alerting on a line as it is printed, a log pipeline is the right tool.

A container’s output is freeform, and that has a consequence worth saying plainly: whatever your application prints is what lands in the table. If it logs a token or a customer email, that string is already in the log file and it will be in the column too. There is no header map to exclude the way there is for a web server log — treat the resulting table with exactly the care you would treat the file.

Two smaller limits, named rather than hidden. On a docker logs export the container’s own line is kept whole in one message column; re-parsing that inner line by its own format is not built yet. And the optional attributes Docker can attach with a logging tag are recognised by shape only — no container in the captured corpus used them, so that path is declared, not proven.

The rest follows the usual design. The Agent connects outbound and reads files; nothing is installed in the container, nothing is written back, the Docker socket and API are never touched, and access is read-only. The retention field starts at 30 days for this preset and is yours to change.

From connection to useful results.

  1. Install the Network Agent on the container host, where it can see the daemon's log directory.
  2. Point the connector at it. Every container's json-file log is matched, and you can narrow that with patterns.
  3. The agent resolves each 64-character directory back to the container it belongs to - real name, image and Compose service - so the rows are readable before you write any SQL.
  4. Every line becomes a row, with its stream (stdout or stderr) and the daemon's timestamp, and rotation is followed as the daemon rotates.
  5. Query it. Errors across every container at once, one service in isolation, or a before-and-after around a deploy.

Put your Container Logs results to work.

One connection serves your spreadsheets, apps and AI tools. Share permission to run a saved query; recipients see the allowed results, with the SQL and source login hidden.

For your team, vendors and partners

Ask Nova about Container Logs

Ask a question in plain English. Nova already knows your Container Logs tables, writes and runs the SQL, explains the answer and charts it — no query to write, no credentials to hand over.

How Nova works

Container Logs to Data Viewer

Share a live Container Logs result with anyone — a colleague, a vendor, a partner — as a bookmarkable grid that re-runs fresh when opened, on a phone, a tablet or a desktop. They see the results, never the SQL or the files.

How the Data Viewer works

For apps and systems

Container Logs REST API

Publish a Container Logs query as an authenticated JSON endpoint any application can call, with an OpenAPI 3.1 spec and ready-made Postman, Insomnia and Hoppscotch collections. No database port is opened.

Explore REST API

Container Logs to Power BI

Paste the generated Power Query M into the Power BI Advanced Editor and the report reads live Container Logs results over HTTPS — no ODBC driver, no database port opened.

Explore Power BI

For AI tools

Delivered to people

Join Container Logs to the rest of your data.

Bring databases, business APIs, files and logs into one read-only query. Every connector supports Federated Queries.

Name the connections and tables. Query Streams handles the source queries and combines the results.

Explore Federated Queries
Illustrative cross-source query
-- nothing copied, nothing merged, nothing scheduled
SELECT   c.region, COUNT(*) AS orders, SUM(i.amount_due) AS invoiced
FROM     containerlogs.fileset.orders1 f
JOIN     pg_crm.public.customers2      c ON c.id = f.customer_id
JOIN     billing.stripe.invoices3      i ON i.customer = c.stripe_id
GROUP BY c.region
ORDER BY invoiced DESC;

Use your own connection, schema, table and column names.

Shared across the connector family.

Every log connector also gives you

  • events — one row per parsed line, with the timestamp, the fields the grammar named, and _source_file naming the file it came from.
  • events_raw_lines — lines the grammar did not match, kept verbatim rather than dropped, so a format you did not anticipate is a question you can still answer.
  • files_current and files_events — the live inventory and the audit trail: what appeared, what rotated, what vanished, and anything held back, with the filename and the reason.
  • Rotation is understood, not worked around — ten named rotation patterns plus plain unrotated files, so yesterday's log and its compressed archive are the same table as today's.

Good to know

  • Read-only, enforced — one statement at a time, SELECT and friends only. Your logs are never written, moved, rotated or truncated by us.
  • Nothing is uploaded — lines are parsed and cached, encrypted, on your own machine beside the agent. Only the result of a query leaves your network.
  • Only new bytes are read — the agent resumes where it left off in each file rather than re-reading it, so a busy log does not cost more every day.
  • The SQL dialect is DuckDB — the same SQL you would write against any other connector.

Questions about Container Logs

Where can I use Container Logs query results?

Use Container Logs results in Excel, Google Sheets, the Data Viewer, your apps through the Database REST API, and AI tools through MCP or Nova. Saved results also feed supported Automation destinations, reports and alerts. Destination availability and allowances depend on your plan.

Do I need to open an inbound port for Container Logs?

No. Install the Network Agent on a machine that can reach Container Logs. The agent connects outward to Query Streams; no inbound port or VPN is needed. Your source must still allow the agent machine to connect under its own access rules.

Are my Container Logs files uploaded?

The Network Agent reads files where they are and keeps its cache on the agent machine. Source files are not uploaded, moved or rewritten. Only requested query results leave your network over the outbound encrypted connection.

What do I need to connect Container Logs?

Folder: The daemon's log directory on the container host, or a folder holding exported logs. Sources: Docker json-file logs, saved docker logs exports, and Portainer's server log. Identity: Container name, image and Compose service resolved from the daemon, so rows are not keyed by a 64-character id. Rotation: Followed as the daemon rotates, plus the ten named rotation patterns for exported files

Can I join Container Logs to other connected sources?

Yes. Federated Queries support every connector type, including databases, business APIs, files and logs. Name the connections and tables in one read-only query. Query Streams translates the source portions as needed and combines their results. See how Federated Queries work.

Do I have to change my Docker logging driver?

No. json-file is Docker’s default driver, and it is the supported one. If you have never configured logging, the files are already there and already in the right shape.

If you have moved to a different driver, the docker logs --timestamps export is the route in: it works whatever the driver, because the CLI reads through the daemon.

How does it know the container’s name and not just its id?

Docker writes its own record of a container in the same folder as the log. Query Streams reads the name out of that file and uses it as the identity column, with the image and the Compose project and service kept as columns beside it. The 64-character folder id is retained as container_id, so nothing is lost.

If that record is missing, unreadable or does not carry a name, the folder name is used instead and the sweep counts how often that happened — so a fallback shows up as a number you can look at, not as a silent substitution.

I am on Docker Desktop for Windows or Mac. Does this work?

Not against the json-file logs directly. Docker Desktop keeps those inside its own virtual machine, and an Agent running as a service on the host cannot see into it.

The way through is the export profile: docker logs --timestamps <container> redirected to a file, in any folder the Agent can read. That is a supported format in its own right, not a workaround — it is the same route people already use to keep a copy of a log after a container is gone.

Is this real-time?

It is on-demand rather than streaming. The files are read as they stand when the query runs, so new lines appear as your containers write them and the file is picked up — not on a fixed nightly schedule, but not a live tail either.

For asking a question of the last few weeks across every container and getting the answer into a spreadsheet, that is the useful shape and considerably less machinery than a log stack.

What about Kubernetes or Podman?

Neither has a profile of its own today, and we would rather say so than imply coverage. What does apply is the export format: the profile is keyed to the exact shape docker logs --timestamps writes — an RFC 3339 nanosecond stamp, a space, then the container’s own line, one file per service. Tooling that writes that same shape reads with it.

The reliable way to find out for your own files is Test connection, which reports what it recognised before you commit to anything.

My containers log JSON. Do I get their fields as columns?

Yes, for json-file logs. The daemon’s envelope is unwrapped and the container’s own JSON inside it is flattened into columns — level, logger, message, host and service where they are present. Anything that does not map to a named column is kept in a data column rather than discarded, so an unusual field is still queryable.

The one thing not taken from the container’s JSON is the timestamp. That comes from the daemon, because an application’s own stamp can be in any format and getting it wrong silently misplaces the row in time.

What about a line the parser cannot read?

It is kept, not dropped. A line that does not match the pinned format is retained as a raw row with a reason attached, so a malformed entry or an unexpected format change shows up as something you can see and query rather than as a silent gap in your numbers.

Put Container Logs where the work happens.

Start with Free: every connector type, 2 data sources and 1 Network Agent. Add plan allowances as your needs grow.

Get the free planSee plan allowances