Elixir/Ports and external process wiring: Difference between revisions

Line 1:

~~This~~ deceivingly simple programming adventure veers unexpectedly into piping and signaling between unix processes.

A deceivingly simple programming adventure veers unexpectedly into piping and signaling between unix processes.

== Context: controlling "rsync" ==

{{Project|source=https://gitlab.com/adamwight/rsync_ex/|status=beta|url=https://hexdocs.pm/rsync/Rsync.html}}

My exploration begins while writing a beta-quality rsync library for Elixir which transfers files in the background and ~~can monitor progress~~. I hoped ~~to learn better~~ how to interface with long-lived external processes—and I ~~got~~ more than I wished for.

My exploration begins while writing a beta-quality rsync library for Elixir which transfers files in the background while monitoring progress. Rsync is the best tool for this since it can resume incomplete transfers and synchronize directories efficiently and it's complex enough that nobody will reimplement it in pure Erlang. I had hoped that this project would teach me how to interface with long-lived external processes—and I learned more than I wished for.

[[File:Monkey eating.jpg|alt=A Toque macaque (Macaca radiata) Monkey eating peanuts. Pictured in Bangalore, India|right|300x300px]]

Line 11:

System.shell("rsync -a source target")

</syntaxhighlight>

This has a few shortcomings, such as the static filenames—it feels unsafe to even demonstrate how string interpolation like <code>#{source}</code> could be misused so let's skip ~~straight~~ to ~~the next tool,~~ <code>System.cmd</code> which doesn't expand its argv:<syntaxhighlight lang="elixir">

This has a few shortcomings, such as the static filenames—it feels unsafe to even demonstrate how string interpolation like <code>#{source}</code> could be misused to make this dynamic so let's skip ahead to how to <code>System.cmd</code> which is safer because it doesn't expand its argv:<syntaxhighlight lang="elixir">

System.find_executable(rsync_path)

|> System.cmd([~w(-a), source, target])

</syntaxhighlight>~~This is safer,~~ but the calling thread loses control and gets no feedback until the transfer is complete.

</syntaxhighlight>Better but the calling thread loses control and gets no feedback until the transfer is complete.

To run a external process asynchronously we reach for Elixir's ~~lowest~~-level <code>Port.open</code> which maps directly to ERTS <code>open_port</code><ref>https://www.erlang.org/doc/apps/erts/erlang.html#open_port/2</ref>. These are tremendously flexible, here we demonstrate ~~turning~~ a few knobs:<syntaxhighlight lang="elixir">

To run a external process asynchronously we will reach for Elixir's low-level <code>Port.open</code> which maps directly to ERTS <code>open_port</code><ref>https://www.erlang.org/doc/apps/erts/erlang.html#open_port/2</ref>. These functions are tremendously flexible, and here we demonstrate how to turn a few knobs:<syntaxhighlight lang="elixir">

Port.open(

{:spawn_executable, rsync_path},

Line 50:

; <code>--itemize-changes</code> : list the operations taken on each file

; <code>--out-format=FORMAT</code> : any format using parameters from rsyncd.conf's <code>log format</code><ref>https://man.freebsd.org/cgi/man.cgi?query=rsyncd.conf</ref>

}}

@@ Line 1: / Line 1: @@
-This deceivingly simple programming adventure veers unexpectedly into piping and signaling between unix processes.
+A deceivingly simple programming adventure veers unexpectedly into piping and signaling between unix processes.
 == Context: controlling "rsync" ==
 {{Project|source=https://gitlab.com/adamwight/rsync_ex/|status=beta|url=https://hexdocs.pm/rsync/Rsync.html}}
-My exploration begins while writing a beta-quality rsync library for Elixir which transfers files in the background and can monitor progress.  I hoped to learn better how to interface with long-lived external processes—and I got more than I wished for.
+My exploration begins while writing a beta-quality rsync library for Elixir which transfers files in the background while monitoring progress.  Rsync is the best tool for this since it can resume incomplete transfers and synchronize directories efficiently and it's complex enough that nobody will reimplement it in pure Erlang.  I had hoped that this project would teach me how to interface with long-lived external processes—and I learned more than I wished for.
 [[File:Monkey eating.jpg|alt=A Toque macaque (Macaca radiata) Monkey eating peanuts. Pictured in Bangalore, India|right|300x300px]]
@@ Line 11: / Line 11: @@
 System.shell("rsync -a source target")
 </syntaxhighlight>
-This has a few shortcomings, such as the static filenames—it feels unsafe to even demonstrate how string interpolation like <code>#{source}</code> could be misused so let's skip straight to the next tool,  <code>System.cmd</code> which doesn't expand its argv:<syntaxhighlight lang="elixir">
+This has a few shortcomings, such as the static filenames—it feels unsafe to even demonstrate how string interpolation like <code>#{source}</code> could be misused to make this dynamic so let's skip ahead to how to <code>System.cmd</code> which is safer because it doesn't expand its argv:<syntaxhighlight lang="elixir">
 System.find_executable(rsync_path)
 |> System.cmd([~w(-a), source, target])
-</syntaxhighlight>This is safer, but the calling thread loses control and gets no feedback until the transfer is complete.
+</syntaxhighlight>Better but the calling thread loses control and gets no feedback until the transfer is complete.
-To run a external process asynchronously we reach for Elixir's lowest-level <code>Port.open</code> which maps directly to ERTS <code>open_port</code><ref>https://www.erlang.org/doc/apps/erts/erlang.html#open_port/2</ref>.  These are tremendously flexible, here we demonstrate turning a few knobs:<syntaxhighlight lang="elixir">
+To run a external process asynchronously we will reach for Elixir's low-level <code>Port.open</code> which maps directly to ERTS <code>open_port</code><ref>https://www.erlang.org/doc/apps/erts/erlang.html#open_port/2</ref>.  These functions are tremendously flexible, and here we demonstrate how to turn a few knobs:<syntaxhighlight lang="elixir">
 Port.open(
    {:spawn_executable, rsync_path},
@@ Line 50: / Line 50: @@
 ; <code>--itemize-changes</code> : list the operations taken on each file
+; <code>--out-format=FORMAT</code> : any format using parameters from rsyncd.conf's <code>log format</code><ref>https://man.freebsd.org/cgi/man.cgi?query=rsyncd.conf</ref>
 }}