StreamWeaver Modules

August 2015 (nothing new since 2011)

OVERVIEW

This note describes the "StreamWeaver" modules included with the full "MusicWeaver" suite. The Weaver main program itself has its own documentation

The StreamWeaver modules give you a 'visual analogue' of unix-style pipes, using the Weaver diagram manager program, into which you can place and connect instances of the supplied modules. The 'StreamView' module provides for input and output of editable text data in a window. Others — 'ReadFile', 'GetFile' and 'WriteFile' — let you pass file data directly to and from pipelines. The 'PipeStream' module will run any shell command line; you can set up any number of these in a diagram, connected together with 'pipelines' that feed the output from one module to the input of another.

Remember that — like all Weaver modules — these modules only respond to (and generate) data packets of a relevant type, in this case 'text' and 'byte data'. They will ignore all others, such as the MIDI packets passed between MusicWeaver elements. Only the MusicWeaver modules that generate text data, such as "MidiMon", or the two that convert between MusicWeaver MIDI and byte-stream MIDI (MidiStream and StreamMidi), can be usefully connected to them.


MODULES

StreamView

Each StreamView element may be used for either or both input and display of text. Lines of text typed in will be sent out the 'Output' connector. Data arriving on the 'Input' path will be displayed. You can also select any segment of the displayed text and transmit it on the output.

This is a multiconnector module: the input stream can be passed through to a succeeding element on a separate output, distinct from the one on which typed or selected text will appear. A second input connection (also passed through) will trigger sending of currently selected displayed text if any packet arrives there (probably not very useful, but it's there if you want it...).

The buttons at the top of the window should be fairly obvious. The left hand one will send any selected text; the next sends an 'End-Of-Data' signal, which will terminate the command in any PipeStream that receives it unless that is set to ignore these. If a StreamView receives an End-Of-Data, it displays a separator line (but takes no other action). Clicking "All" selects the complete text, typically for transmission or deletion.

The StreamView window behaves like a standard BTextView, except in one respect: in 'Lines' mode the <ENTER> key only inserts a newline in the window when it is at the very end of the text, but it always sends the entire line. This lets you easily resend any line by moving the cursor anywhere within it and typing <ENTER>, without disturbing the displayed layout. <Ctrl-ENTER> has essentially the reverse effect: it always inserts a newline, but never transmits the line. This non-standard behaviour is only seem in 'Lines' mode (see below). (The display text is not saved, by the way, when you save the diagram.}

The 'Send as:' selector at top right of the panel determines which user actions result in data being transmitted. In the default state('Lines'), typed-in data is sent only when <ENTER> is typed, as a complete line. In 'Keystrokes' mode it is sent a character at a time, keystroke by keystroke. (Keystroke mode is only effective on typed-in data — it doesn't change how a selected block is sent.) When the mode is 'Selected Only', nothing is ever sent as characters are entered or a new line is started: you must highlight the data you want to send and click 'SEND Selected'. In these latter two modes, the panel accepts input in a perfectly standard way — <ENTER> is always echoed in the window. Also remember, though, that in Keystroke mode everything is sent, including things like cursor movement and deletions.

You can of course paste text from somewhere else into the view, but it will never be sent directly, even if it contains newlines. To transmit it, you either have to move the cursor to the line and press <ENTER> or select it and use the Send button.

At the time of writing, the Haiku BTextView class seems to have a selection bug that may show up if you have edited the text. If you select a segment of text for transmission that you have previously edited, the output may contain characters you think you deleted, or not show ones that ought to be there! If you hit this problem, copy and re-paste the line you see in the view, and you should then get the correct output.


WriteFile

This is a Filter that writes an incoming data stream to a file selected by a standard File Save Panel. This is opened with the top button in the element's control panel. You can't append to an existing file (though you could concatenate streams to a WriteFile via a PipeStream with command 'cat' and 'Ignore E-O-D'!).

While a file is open to accept data, its name is displayed in the control panel. It will remain open until an End-Of-Data is received or the 'CLOSE' button is clicked. If no file is open, the panel displays '<Closed>'. The current directory in the filepanel, though not any filename, is recorded in the configuration file when you save.


ReadFile

This is a Source module that will stream (text or byte) data from a file (or files) onto a Weaver path. You can either just drag the desired file(s) onto its panel, or click the button in the panel to bring up a File Selector Panel and choose them there. The files are sent immediately, and each is followed by an End-Of-Data signal.

For convenience, the File Selector Panel remains visible once it is opened, so you can quickly transmit several files in sequence. If you want to send multiple files in a particular order, you should drag (or select) them in that order separately, rather than as a group, as the order in which a group of files is processed is up to the Tracker's fancy alone.

The current directory in the filepanel is recorded in the configuration file when you save, but the filename itself is not.


GetFile

This has a similar purpose to ReadFile, except that, instead of reading files from storage on demand, it reads a single file into memory and can spit it out repeatedly.

At the top of the control panel is a text field where you can directly type in or paste a full pathname (the default folder is always /boot/home). More simply you can drag a file onto the panel, or browse with the "Select..." button. The current file is always shown in the text field. If the "Send when set" box is checked, the file will be sent immediately it is loaded. Clicking the "Send" button will always transmit it.

Unlike ReadFile, the pathname is recorded in the configuration, so it will be reloaded (and sent if desired) when the configuration is.


PipeStream

Each instance of this module can be supplied with any valid shell command line, which it can then execute repeatedly. It can be set to a particular directory if desired. Standard output from the command appears at the 'Output' connector of the element; data arriving on an 'Input' path get passed (without any interpretation) to the command's standard input.

This is a 'Multiconnector' element. The first output path connected is by default the command's standard output, but a second one may also be attached to get a 'pass-through' of the input stream. A second input connection (also passed through) will trigger (re-)execution of the command when any packet arrives there (same as the 'Run' button).

The element control panel has two text entry fields: one for the desired execution directory, and the other for the command itself. The directory field is unimportant unless the command accesses or creates files (or unless the command itself is not in the default shell path); if you leave it blank, it will be taken as the system default — presumably /boot/home. (Changing the directory while a command is running has no effect until a new run is started.) To browse for a directory — friendlier than typing it in! — use the 'Browse Dirs' button on the right to bring up a selection panel. Or you can simply drag a folder (or a file within it) onto the control panel to select it. ["Directory" is used preferentially to "Folder" here because that is more common command-line/shell usage.]

Typing <ENTER> after typing the command line will start execution immediately (and the displayed status will change from 'Stopped' to 'RUNNING'). You can also use the 'Run' button to begin execution. You have to use this if you want to run the same command again, because the text-entry control ignores the <ENTER> key if the line has not been changed. (Caution: clicking 'Run' when the command is already running will cause it to stop rather than run. The 'Trigger' input will restart a running command though.)

Depending on the command you enter, it may finish its own execution and terminate, or it may terminate when it receives 'end-of-data' on its input, or it might just keep on going until forcibly killed. If you need to kill it, the 'Stop' button should do the job — but see below for variant behaviour. When a command terminates, an end-of-data is always sent to the output path (whether or not there has actually been any output).

The popup-selection button at top right of the panel controls the run mode. 'Normal' means that the command will just run once each time it is started; if it is processing standard input, it will (if well behaved.) quit when it gets an end-of-data. If you would rather that end-of-data signals on the input stream did not get passed to the command (so that it will keep on running until stopped) set the selection to 'Ignore E-O-D'. If you want it to get restarted automatically each time it terminates, use 'Auto-Restart'. Response to the 'Stop' button is slightly different in this last mode: the first click of the button just switches the mode back to 'Normal', so that the command will complete its processing if possible; if you need to really shut it down, click it a second time.

When you save a diagram to a configuration file, the state of the element is preserved in that file. The command and directory are recorded (as text strings), as is the mode setting and whether or not the command is actually running when the save is performed.

The module only handles standard input and standard output. It does not, for example, directly handle standard error output. If you want to see this on the Output connector of the element, just redirect with the '2>&1' bash convention. Or of course you can redirect it elsewhere to a file or a named pipe.

As there is only one input stream, you cannot easily use commands such as 'diff' that by nature need two inputs, except again by using files or named pipes [which sort of seems to negate the 'visual pipeline' concept!].


PacketMon

This simply outputs text lines reporting the contents of each Weaver packet that arrives on its input. Connect its output to a StreamView to read it. It is only really intended for internal diagnostic use, but is included in case you want to glimpse the internals...

The items in a packet are shown with their ID tags, their size in bytes, and the content of the data field — either a pointer to data if size is non-zero, or the data itself for zero size. All items before the terminating dashed line are in a single packet.


Copyright and Distribution

This documentation, the Weaver program, and all associated modules, are Copyright 1997-2015 by Peter J. Goodeve. All rights are reserved. The StreamWeaver package is, however, freely distributable.

You are permitted to distribute the Weaver program, the StreamWeaver modules, and documentation in the form of the supplied ZIP archive without charge for non-commercial purposes, provided that the whole package is kept intact. For commercial use or distribution (other than charging a reasonable media and copying fee) please contact the author.



Author:

				Pete Goodeve
				Berkeley, California

        e-mail: pete@GoodeveCa.net
                pete.goodeve@computer.org