I'll join the chorus praising socat, but also share my gratitude for Gerhard, the developer / maintainer.
We exchanged emails back and forth about a use case I had, which was to split a serial device into two PTYs, allowing a process to control one while I monitor the other with picocom. We couldn't get it to work, so I gave up, but then Gerhard wrote back a few months later saying he added it as a feature.
It can be frustrating to figure out the right incantation for some advanced use cases, but this is more a complaint about esoteric corners of the operating system than socat which exposes the full complexity to you.
One of the invocations I recently worked out was to wrap an interactive process with a PTY, so that I could make a mock serial device that emulates a piece of hardware.
Nearly all of the microservices we develop have a way to be run locally with all dependent services using docker-compose.
The problem is most teams don't provide a way to do efficient local development for their particular service. Some do remote debugging into the built/running container (slow and requires rebuilding docker image for every change). Some inspect logs to debug (yikes!).
When I want to debug a particular service in IntelliJ, I simply remove it from the docker network and replace it with a container running socat, forwarding the appropriate port to host.docker.internal (the host machine). This is on a MAC, of course, where Docker runs in a VM.
This allows me to do in-IDE debugging of any service that runs in the docker network without having to rebuild the image every time I make a change.
I know there are IntelliJ plugins that provide a similar workflow, but I believe they all build an image and deploy a container to the network for every change. I could be wrong.
> remove it from the docker network and replace it with a container running socat, forwarding the appropriate port to host.docker.internal (the host machine
Would you happen to have a fully worked example of this? We to have some services that we can and do run via docker compose - but it's often a cumbersome process to move one and one service out of the docker compose setup for debug/devel.
A similar tool: spiped[0]. Connects two arbitrary TCP4, TCP6, or UNIX socket addresses, encrypting + authenticating data using a preshared key. For many applications it's basically socat + security.
Have they done any audits or had critical analysis of their security code? These days I'm super hesitant to consider anything that hasn't had a lot of eyes on it. It's just too easy for a tiny screwup in protocol design/implementation, random number generation, etc. that puts everything at risk. Why risk it when I can just use wireguard.
cperciva is the author of spiped and a security expert [0].
But your question about an audit is still a good one! Tarsnap has bug bounties, but the top hit for “spiped security audit” led to a sort of amusing flame war on HN [1].
Lots of people have looked at the spiped source code; it has been around since 2011 and I've even handed out a few bug bounties.
It's also considerably simpler than wireguard, which reduces the potential for vulnerabilities; and operates on a per-connection basis, which means you can do things like establishing a secure connection to another host but have a socket endpoint which can only be accessed by certain users.
"lots of people looking at source code" is contemporarily a weak argument against audit and review by experts
I would absolutely love to get some deliberate code review from experts. I know tedu has looked at some of the code -- he sent me some minor corrections -- and I would certainly count him as an expert, but I don't know if his review was systematic or if he just glanced at a handful of files.
Socat is the default implementation for Kubernetes port-forwarding [1] when using dockershim for the kubelet runtime. Similar to how tar is used as the default Kubernetes cp command implementation.
I often use a socat docker container to proxy network traffic to docker containers. It effectively allows creating dynamic docker port mappings.
For example, I have a database with no docker port mappings (only a web application container in the same docker network accesses it directly). I can use a socat container with port mappings to proxy network traffic to the database.
It is exposed through docker, just via a separate container. The reason for doing so is that a container can't have new port mappings added after it is created.
Using the socat container allows me to access a port on my original container that I wouldn't be able to access unless I recreated that container.
As for why the database didn't have port mappings originally, it is best not to allow network access to things unless it's necessary. It's the principle of least privilege.
I've used socat in the past to debug all kinds of odd network behavior. It helped me discover a networking issue with some code by comparing performance of piping a Unix socket over TCP using socat vs. the native TCP implementation, which turned out to be caused by an overly conservative default send buffer limit.
socat is an invaluable tool. I first came about it at my first job, where we (just two clueless dudes) had to migrate the mostly undocumented infrastructure of about 15 machines at Digital Ocean to Linode, adding replication and failover on top of that (making it ~35 or so machines).
During that migration, we tried to minimize downtime, so we made sure that requests made to the old infrastructure were aptly forwarded to the new infrastructure wherever possible, until DNS properly propagated. I found out about socat and we went for it. It worked great, and I marveled at the simplicity of the solution!
Ever since then, I've used socat both as an emergency saviour and as an actual service. I seem to recall using it once to go around some docker bug or shenanigans.
Whenever someone mentions netcat or nc I always ask "which one?". There are at last count 3 different implementations, all subtly incompatible. Which is another reason why socat is better since there's only one to deal with.
Although it is "freely given away to the Internet community" with "an obligation to give credit where due", at least OpenBSD and GNU have seen the need to write their own versions under their project licenses:
(The OpenBSD version has been ported to at least FreeBSD and Apple Macintosh OS.)
All of them have the same basic telnet's `host port` syntax for outbound TCP connections, but annoyingly the syntax for opening a local listening TCP socket varies. Say, you want to open a TCP socket listening on port 1234 (local), and a confirmation when it is ready:
The original and GNU netcats: netcat -v -l -p 1234
Indeed. There are examples for making local/remote shells. It can make a pty, use setsid() to be a daemon, reset the terminal state, then listen(), etc, all in a one liner. Or route a serial port over ip via ptys in raw mode, again, with a one-liner.
I followed the directions at https://stuartleeks.com/posts/wsl-ssh-key-forward-to-windows... to get the combination of socat and npiperelay to work for this purpose. I did slightly deviate from his final script by using pgrep instead of the combination of ps and grep to detect if npiperelay was already running.
Edit: These directions are very similar to those linked in the comment this is in reply to and either will work.
Just to throw another one of these bridge applications into the mix, here is one that allows Pageant apps to communicate with Windows' native ssh-agent.
When I looked for a similar solution quite a while ago one didn't exist so I wrote my own. I have been meaning to clean the code up and post it for about 18 months now but have never gotten around to it. Need to check this out and see if it is worth still using what I wrote.
Let that be a lesson to not abandon your side projects :)
I was in need of a solution to this issue for a long time before ndbeals thankfully released this about 9mo ago. And I was checking around the web regularly! I nearly wrote my own as well.
The use case for me was to allow WinSCP and Sourcetree to be used with the Windows ssh agent.
WinSCP is the reason I wrote mine. I'll probably stick with the one I wrote since it is win32 native and seems to consume less resources. I'm not a developer by trade and barely one by hobby. This was the first and so far only win32 app I have written so the code while functional is very ugly. I'm sure anyone familiar with win32 programming would be horrified. I should just throw it over the wall as is. Obviously the cleanup I was going to do before releasing it is never going to happen.
I use socat a lot, especially when wanting to transfer files without incurring the ssh encryption overhead or rsync daemon setup.
Just tar | scocat and voilà,
Superfast whole directory tree transfer.
Sprinkle a bit of mbuffer or openssl on top and you can do a lot!
It talks serial too. I've used socat to send out data to the network from a set of serial connected devices for environmental modelling. Ended up being one of those jobs I got done from end to end in an afternoon with the socat part taking 10mins
Oh! I might as well use it for line editing on serial line with the readline local and serial remote addresses, for example when talking AT to a modem. Great! :)
And with history and all that stuff, too! I guess it can be used to forward serial port over network too.
Is it possible to use Socat (or any utility) to redirect a UDP connection on a local Windows machine to TCP port 443/80 to a remote system listening on 443/80 that will connect to the original intended UDP connection?
I can also do port redirects with iptables that can assist with solving the problem.
I need to host a microservice that uses UDP which won’t connect out on most corporate networks due to firewalls. I need to get the connection out via a standard TCP port.
Nothing malicious. It’s a basic corporate client/server app, but it’s bound to a UDP port that I can’t change.
huh. I haven't heard of this one. Seems like it's been around a while? Why might I use this compared to something more well known? haproxy doesn't seem to do generic udp, but maybe linux virtual server or something like that?
We use socat to buffer logs sent from containers (docker-compose) during the short window at startup where logstash and elasticsearch are not ready yet. Works great!
Similar: using it to get syslog out of self-chrooting ssh with minimal trauma:
socat -u UNIX-RECV:/home/sftponly/user/dev/log,mode=666 UNIX-SENDTO:/dev/log
It's also an CLI IMAPS client, with history:
socat READLINE,history=$HOME/.imaps_history EXEC:'"openssl s_client -connect mailserver:993"'
or a CLI web browser with history:
socat -d -d READLINE,history=$HOME/.http_history TCP4:www.domain.com:www,crnl
or, if you have to interact with something that has no readline, say sendmail:
socat READLINE EXEC:"sendmail -bt"
I made a small utility that'll replay socat log files captured by the -lf -x -v flags with the timing it collects if used in this way. It is very simple and is basically only there if you can't be bothered to roll your own:
https://gitlab.com/inivekin/socatplayer/-/releases
So if you're commonly doing a socat process like:
socat -x OPEN:/dev/ttyS0,b115200,echo=0,icanon=0 PTY,link=/dev/ttyO0,rawer 2> socat_log.log
When used to establish a persistent connection to a remote listening (TCP) port, it has NO WAY to detect that the remote end has dropped the connect, and automatically reconnect. It just does not work.
Yes, I know all about all the keep-alive and retry options in the manual. Try it yourself.
It's one of those horrible tools that you can get up and running quickly, but can never make work reliably.
We exchanged emails back and forth about a use case I had, which was to split a serial device into two PTYs, allowing a process to control one while I monitor the other with picocom. We couldn't get it to work, so I gave up, but then Gerhard wrote back a few months later saying he added it as a feature.
It can be frustrating to figure out the right incantation for some advanced use cases, but this is more a complaint about esoteric corners of the operating system than socat which exposes the full complexity to you.
One of the invocations I recently worked out was to wrap an interactive process with a PTY, so that I could make a mock serial device that emulates a piece of hardware.