Linux on Monish Kumar's Blog

Barbwire: an eBPF based behavioral correlator

Sun, 08 Mar 2026 16:42:18 +0530

Introduction

After the eBPF learning phase I wrote about in the last post, I wanted to build something that looked like a real security tool. Not because I thought I’d ship it, but because the programs I tried before where stupid. At some point you need a problem with actual constraints.

Barbwire is that project. It’s a lightweight behavioral correlator for Linux. It watches for processes that open sensitive files and then make network connections within a short time window. This is classic exfil behavior and rough fingerprint for credential harvesting and so on. It’s not a production EDR. It’s an experiment. It’s on my Github if anyone’s interested.

The core design decision

The first instinct when building something with eBPF is to put logic in the BPF side. It’s running in the kernel, it’s fast, why not do the correlation there?

I thought about it and decided against it early. The BPF verifier gets unhappy with complex logic. It’s harder to debug, less portable across kernel versions, and for this use case, unnecessary. Userspace is fast enough. The design I landed on: BPF is a dumb data collector, userspace does the thinking.

The BPF side attaches tracepoints to three syscalls. sys_enter_openat for file opens, sys_enter_connect for network connections, and sys_enter_execve for process execution to track lineage. All three write into a single shared ring buffer with an event type field. Userspace reads and routes.

I used a ring buffer over a perf event array deliberately. Perf event arrays are per-CPU, so you have to poll multiple buffers and deal with out-of-order events. A ring buffer is a single shared buffer, better throughput, simpler consumer. So obviously I wanted the path of least resistance.

How correlation and scoring works

On the userspace side, three in-memory maps hold state. Recent file opens per PID, process lineage from exec events, and a deduplication map to avoid alerting on the same PID repeatedly. A cleanup goroutine evicts stale entries every 30 seconds.

The correlation window is configurable. I have it set to 5 seconds. When a connect event comes in, Barbwire looks back at that PID’s recent file opens and checks if any fall within the window. If they do, it scores the pair.

Scoring is based on file and connect pairs, not individual signals. A process connecting to the network is normal. Opening /etc/passwd on its own is normal. Both within 5 seconds is suspicious. The score comes from which file category was accessed:

suspicious_files:
  - category: "credential access"
    base_score: 3
    patterns:
      - "/etc/passwd"
      - "/etc/shadow"
      - "/.aws/credentials"
  - category: "ssh key exfiltration"
    base_score: 3
    patterns:
      - ".ssh/id_rsa"
      - ".ssh/id_ed25519"

Process lineage modifies the score up or down. A shell spawning a process that reads credentials and connects out is more suspicious than the same behavior happening under systemd:

suspicious_parents:
  - comm: "bash"
    modifier: 2
  - comm: "sshd"
    modifier: 3

legit_parents:
  - comm: "systemd"
    modifier: -2
  - comm: "dockerd"
    modifier: -2

Alert threshold and severity thresholds are decoupled. The threshold decides if the alert is an actual alert, severity is a classification of the alert. When a connect event comes in, Barbwire picks the highest scoring file match for that PID and emits one alert. One alert per connect event per process, no duplicates (hopefully).

A fired alert looks like this:

┌─ barbwire alert — PID 41890  ─────────────
│  command  : python
│  file     : /etc/passwd
│  connect  : 2404:6800:4007:821::200e:80
│  severity : HIGH
│  reasons  : credential access, suspicious parent: fish
│  parent   : fish (pid 13387)
│  gparent  : tmux: server (pid 9483)
└─────────────────────────────────────────────

My Screwups while building it

A few things broke in ways that were actually instructive. The first one was actually two bugs hiding behind each other, which made it genuinely annoying to debug.

There was a constant mismatch between the C and Go sides. EVENT_EXEC, the event type for sys_enter_execve tracepoint, was 2 and EVENT_CONNECT, the event type for sys_enter_connect, was 3 in the BPF code, but I had them swapped in Go. At the same time, I had attached the exec tracepoint and called Close() on it immediately without a defer, so it detached right after attaching (Yeah this is embrassing).

The symptom was simple, in terms of events I had none. Thing was execve events were being routed to the connect handler. Since those events carry no IP address family, they got dropped as invalid. Meanwhile connect events were being routed to the exec handler, which was already closed. Everything was failing quietly in a different place than where I was looking.

I spent a while staring at the correlation logic before going back to basics and checking whether the tracepoints were even active. Once I caught the missing defer, connect events started showing up, but in the wrong handler. That’s when I found the swapped constants. Two separate bugs, one symptom, neither pointing at the other. Really annoying

The most useful bug involved stale memory in the event struct. bpf_ringbuf_reserve gives you a pointer to uninitialized memory. If you skip the __builtin_memset before writing your fields, whatever was in memory from the previous event bleeds into the fields you didn’t set. I had filenames with garbage characters appended, leftovers from longer filenames earlier in the buffer. The fix is one line. But you have to know it’s needed.

Limitations and takeaways

The biggest problem with Barbwire is the one that can’t be tuned away.

For example, curl reads /etc/passwd on every invocation. Here is it making the call:

Not because it’s malicious, but because that’s how it works. I believe, basically any program using libc networking does this. Barbwire flags it every time. No matter how hard I tuned the knobs, I couldn’t get rid of such alerts. Then it hit me, single-process correlation without broader context has a ceiling.

Evidently, production security tools handle this differently. Binary hash whitelisting instead of process name matching, which is spoofable. Process ancestry graphs that track behavioral chains across multiple processes. Behavioral baselines built up over time. A shell that reads /etc/passwd and then forks a child that connects out would be caught by a graph-based correlator. Barbwire misses it because the file open and the connect are in different PIDs.

Building this made that limitation concrete in a way I don’t think I’d have gotten from just reading about EDR design. Honestly, I never even thought of these before. You also understand why process ancestry graphs exist when your tool fails exactly where they’d succeed.

The other thing I took away: the BPF to userspace boundary is unforgiving. Memory layout, pointer semantics, which helper functions apply where, what the verifier accepts. The documentation covers the what, but the why only shows up when things break.

All things considered, I had fun with all this. That’s the whole point if you think about it.

I wanted to poke at the Linux kernel. eBPF was the answer

Sat, 07 Mar 2026 20:32:58 +0530

Introduction

A while back I came across Cloudflare’s writeup on how they mitigated a 3.8 Tbps DDoS attack, the largest ever disclosed publicly at the time. The post goes deep into how their systems detected and dropped attack traffic autonomously (quite interesting stuff). One part that caught my eye: they used XDP and eBPF to drop packets directly at the NIC level, before the kernel network stack even saw them. I had zero clue on what eBPF was back then. I thought that was interesting, bookmarked it, and then forgot about it (Yeah I do that a lot).

When I eventually got around to it, I was looking to get into lower level Linux stuff anyway. How the kernel works, how you can poke at it from the outside. eBPF kept coming up. I also saw it framed as a safer alternative to writing kernel modules for certain use cases. That part made immediate sense to me, because writing kernel modules means one bad pointer dereference and your machine is just down. So I thought maybe I could look into it seriously.

What even is eBPF

eBPF lets you run sandboxed programs inside the Linux kernel without modifying the kernel source or loading a module. Before your program runs, the kernel verifies it. No unbounded loops, no out-of-bounds memory access, every code path has to terminate. If it doesn’t pass, it doesn’t load. However it doesn’t mean it’s bullet-proof, I still found some bugs and weird things happen now and then. But it catches the most obvious ones.

Evidently, it started as Berkeley Packet Filter, which was just a way to filter network packets efficiently. The “extended” part came later and expanded it well beyond that when people wanted better tracing options when debugging inside the kernel. These days it shows up in observability tools, performance profilers, security products, and a lot of infrastructure tooling that runs quietly in the background of production systems.

How I learned it

I started reading Learning eBPF by Liz Rice and went through it cover to cover. It builds up a solid mental model of how everything fits together, the verifier, maps, ring buffers, CO-RE, without throwing you into kernel internals from page one. Great book, would recommend it if you’re starting out.

After that I spent time staring and reading through examples at libbpf-bootstrap and cilium/ebpf. That was just to see how things are done in real projects and scripts.

Then I started writing stuff. A per-process file access counter. A pure C rough version of what would eventually become Barbwire. None of it was useful or anything. It was just to get familiar with the toolchain, get yelled at by the verifier, figure out why and repeat.

What I liked about eBPF

A couple of things genuinely impressed me.

The first was CO-RE, which stands for Compile Once, Run Everywhere. Despite my reservations on Java, I always thought the idea of Code once, run everywhere was pretty neat (Thanks to JVM). CO-RE brings the same thing to Linux kernel programming. It handles struct field offset relocations across kernel versions, so your program isn’t silently tied to the exact kernel it was compiled against. Without it, anything that walks kernel structs like task_struct breaks the moment you run it on a different machine. With it, the same binary works across versions. It is remarkable as far as I can tell.

The second was how dynamic the whole thing is. With kernel modules you write code, compile, load, and if something goes wrong you restart. With eBPF you attach programs to running systems and detach them without rebooting or disrupting anything. For observability and containers especially, that matters a lot. You can instrument a live system, see what you need, and pull the probe out. No downtime, no risk of leaving a bad module loaded.

What came next

After a few weeks of experiments I felt ready to build something with an actual purpose. That became Barbwire, a lightweight behavioral correlator that watches for processes opening sensitive files and then making network connections. I’ll write about that in the next post.

It’s not a production security tool. It’s an experiment. But building it was a good reason to use eBPF for something real instead of just counting file accesses forever.

Miscellaneous - Linux Basics

Sat, 26 Apr 2025 17:51:33 +0530

Linux Basics

Challenge Statement

Author: @aenygma

You’re expected to answer a series of questions to get the flag.

To view the questions, and answer them, you’ll use the answer tool.

Display questions: answer

Answer a question: answer x where x is question number.

Note: This challenge was accompanied with a per-user instance

Solution

Spinning up the per-user instance for the challenge I was able to connect using netcat to a “shell”. Except it wasn’t a shell but a relay of data using socat. Anyway once I got in I was greeted with a message and as per the challenge description I ran answer to see the questions.

The entire challenge is easy if you know your way around a bash shell.

Question 0: What's your home directory?

Using pwd command I checked which directory I am currently at. I was in the home directory of a user named user. So the answer is /home/user

Question 1: Search the man pages. What command would you use to generate random permutations?

I didn’t refer the man pages, but a quick search on internet gave me the utility shuf. It’s one of the core utilities of linux systems apparently. So that was the answer.

Question 2: On what day was /home/user/myfile.txt modified? Use the date format 2019-12-31 Question 3: How big is /home/user/myfile.txt, in kilobytes? Round to the nearest whole number. Question 4: What user owns the file /home/user/myfile.txt Question 5: What's the 3-digit octal permissions of the file /home/user/myfile.txt? (e.g 777)

Now all these questions are centered around myfile.txt in the home directory. So all it takes is the output of ls -l command.

For question 2, the date modified is Aug 29 1997 as seen from the output. Putting it in format required it is 1997-08-29.

For question 3, the size of the file myfile.txt is required. It is the 22200 number that is the size. But this is in bytes. 1 kilobyte is 100 bytes. So 22200 bytes is roughly 22.2 kilobytes. Since they asked to round to the nearest whole number, 22 is the answer.

For question 4, the user that owns is the 3rd column in the output. The next to that is the group that owns the file. But since we just need the user who owns it, it is root user. So answer is root.

For question 5, we need the 3-digit octal permission of the file. We don’t directly have the answer, but we can calculate it from the permissions column which is the first column. The first column is filled with 10 characters. Ignore the first character as it tells us what type of entry it is. The rest of it can be used to derive the octal permission. Each triplet in the 9 characters correspond to one digit in the octal. With r - 4, w - 2, x - 1 and -(dash) - 0. So the octal permission is (4+2+1)(4+1)(4) = 754. That’s the answer.

Question 6: What is the user id of 'admin'?

We can simply use the id command to view the ID related to admin account. So according to the output, it is 1338.

Question 7: There is a user 'john' on the system. Can they write to /home/user/myfile.txt? (yes/no) Question 8: Can the 'admin' user execute /home/user/myfile.txt? (yes/no)

Now again we go back to myfile.txt. So again we just need the output of ls -l for this job.

For question 7, as we can see the ‘w’ is set only for the user. So only the user who owns the file can write to it. So, no, john cannot write to it.

For question 8, the execution flag ‘x’ is set for both user and group who owns the file. And admin happens to be the group. So yes, admin can execute the file.

Question 9: Which user on the system, except for you, root, admin and john, can execute /home/user/myfile.txt?

For question 9, if we recall from question 8, only the user and group who owns the file can execute it. So the user who owns it is root. But what about the group. If anyone other than the given list of user can execute it, they must be a member of group admin. So examining /etc/group, we can notice a user rose in the group admin. So yeah, that user can execute the file.

Question 10: /home/user/myfile.txt looks like a txt file, but it actually isn't. What kind of file is it?

So it is not a text file? When in doubt, use the file utility. It reveals that it is a JPEG image. So the answer is jpeg.

Once I give that answer, the program gives me the flag. So that’s it? Well yes, but no.

My curiosity wanted me to know what that image is. So I exfiltrated it. For anyone wondering, the tool xxd was available. So I dumped the image. Copied the dump to somehexdump and did the reverse operation on my machine to somejpeg.jpeg. Opening the file I found:

Hmm. That seems like something. Something great. Or some great rabbit hole. Or a wild goose hunt. I don’t know. I spend some time snooping around the instance trying to look for things but then I moved on to the next challenge. So if I find anyone who found it (if it is real) I’ll update.