ueconv version 0
================

ueconv is a tool for finding conversation audio data corresponding to given text
data in package files used by Unreal Engine (at least UE1). Additionally, it can
output conversation graphs in DOT format.

For example, with Deus Ex, the conversation package files are named
DeusExConText.u, DeusExConAudioIntro.u, DeusExConAudioMisson00.u, etc.

Note that these instructions are mainly written with the default Windows build
in mind. That is, full (with SDL+SDL_ttf+SDL_image support) build is assumed,
and the instructions will e.g. refer to .dll's instead of .so's, use
Windows-like paths and so forth. Linux users probably know their way around
those bits. Most of the instructions shouldn't be too platform-specific though.

Also note that these instructions will use, even in Windows context,
forward slashes for path separation, like c:/stuff/foobar, instead of
backslashes like c:\stuff\foobar, although the latter way seems more common in
most Windows things. Forward slashes however are allowed in Windows too, and
they're more convenient in a few places where the backslash character is used
as an escape character.

Also, note that ueconv will never create directories that don't already exist.
It'll only create files. You need to create the directories yourself.

------------
PREPARATIONS
------------

Windows
-------

For Windows, pre-built binaries are in directory binary_win. However, you need
to download dll files for SDL, SDL_ttf and SDL_image libraries (unless you
already have them):

- SDL: go to http://www.libsdl.org/download-1.2.php , download runtime library
    for Win32, not the 64-bit version
- SDL_ttf: go to http://www.libsdl.org/projects/SDL_ttf/ , download binary for
    Windows, not the 64-bit version
- SDL_image: go to http://www.libsdl.org/projects/SDL_image/ , download binary
    for Windows, not the 64-bit version

Then extract those and put all the .dll files to the same directory with ueconv
(or to appropriate system directory, if you want the dll's to be available to
other programs too).

Also, if you intend to use the graph visualization mode, install Graphviz:
go to http://graphviz.org/Download_windows.php , get "current stable release".
Doesn't matter where you put it, but you'll need to give the path do dot.exe for
ueconv later.


Linux
-----

For Linux, you need to build from source (in directory "source") with a c99
compiler. If you have (a properly up-to-date version of) gcc and make, you can
just type "make" in the source directory; it'll build into an executable named
ueconv in the same directory. You need SDL, SDL_image and SDL_ttf (and
sdl-config and pkg-config, though you can work around those if you know how) for
the full build; you can build with any of those disabled like e.g.
"make NO_SDL_IMAGE=1" (without quotes), and similarly for NO_SDL and NO_SDL_TTF.
You should use at least SDL and SDL_ttf for the GUI. The GUI-less interface
isn't documented (though if you need it, feel free to poke me and I'll write
something). If you want to build with other than gcc and make, you can probably
figure some things out from the Makefile.

Also, if you intend to use the graph visualization mode, install Graphviz.


For other platforms, I don't know. Also, the code may contain some non-portable
bits (especially certain system() calls).

--------------------------------
MOST RELEVANT USAGE INSTRUCTIONS
--------------------------------

The ueconv program has a graphical interface by default; the non-default textual
interface is not properly described in this file.

First, try launching runme_example.bat . It should launch ueconv with some
hopefully clarifying settings. It's possible that running the .bat will fail,
and if it does, and complains about fonts, you need to modify the font file set
in the .bat file (just edit it with Notepad or something). 

A blue-ish interface with text boxes and buttons should then appear. If text is
too big or small, try closing and re-starting the program, but this time add an
argument font-size=SIZE where SIZE is the wanted font size (e.g. font-size=10 .
Default is 11).

In the topmost text box, you should write the prefix of the package files -
"prefix" here means that if the conversation text file is named
DeusExConText.u , its prefix is DeusExCon . You need to have the *Text.u and
*Audio*.u files all in the same directory. So for example, if you're using
Windows and have Deus Ex installed in c:/games/DeusEx, you should enter
c:/games/DeusEx/System/DeusExCon .

The next three boxes are search parameters:

Part of spoken text: This should be a contiguous part of the line that you want
to find. It's case insensitive, so the texts "RoTtEN", "rotten", "ROTTEN" etc.
will all be treated the same. Otherwise you should enter the text just as it
appears in the game; no extra spaces or anything (the search isn't too smart
that way). Note that at least in Deus Ex, there often seem to be two spaces
after a period.

Part of speaker name: Part of the name of the character that speaks the line.
Case insensitiveness etc. are as described above. Note that this is the
"engine name" (or whatever it should be called), not the actual in-game name;
these are often different. For example in Deus Ex, JC Denton's "engine name"
is JCDenton. Most of the engine names in at least Deus Ex seem to be rather
obvious; just the character's actual name except with the first and last names
written together without spaces. SandraRenton, MorganEverett etc. If you don't
know the engine name of the speaker, you can just leave the box empty, in which
the search won't be restricted by the speaker name.

Part of listener name: Similar to above, but for the person being spoken to.
Again, this too can be left empty.

The "Speech sound output file" is the filename where you want the speech audio
data to be saved. E.g. c:/MyDeusExSpeeches/whatashame.mp3 . At least in
Deus Ex the format is mp3, don't know about others. At the moment there is no
way of making ueconv set the file suffix automatically. NOTE: ueconv won't
create any directories that don't exist, it'll only create the file. So you
need to create the directories yourself.

"Conversation graph DOT output file" is the filename where the conversation
graph is saved, in DOT format, when the "Save conversation graph" button is
clicked. More info later in this file.

The "User-defined external command" box is a bit more complex, and will be
explained later in this file. Same goes for the "DOT command" box.

Once you've written to all the text boxes you want, clicking the "Find matches"
button will search all the matching lines; they'll appear to the box left to the
buttons. The "Save speech" button will save the audio of the currently-selected
line into the specified sound output file.

As an example: to search that specific line from Deus Ex, you can do something
like this:
 - set the package name prefix, e.g. c:/games/DeusEx/System/DeusExCon
 - set spoken text to "what a sh" (without quotes)
 - set speaker name to jcd
 - set listener name to sandraren
 - set speech output file, e.g. c:/MyDeusExSpeeches/whatashame.mp3
 - click "Find matches" (or press shift+enter)
(- verify that the correct line appears)
 - click "Save speech"
 - speech audio should now be in c:/MyDeusExSpeeches/whatashame.mp3

In this case only the one line should be found; generally multiple lines can
match, and you can select it by clicking (and there's the scroll bar and stuff).

----------------------------------
FURTHER MISCELLANEOUS INSTRUCTIONS
----------------------------------

Some things that were not described above:


User-defined external command, and text box parameter patterns
--------------------------------------------------------------

In addition to the text boxes described above, there's also the
"User-defined external command" box. There you can write a command that'll be
executed when the "Invoke user-defined command" button is clicked. The command
will be executed as a command-line command using the C system() function. The
catch is that before executing the command, the program will replace certain
patterns in the command with useful things. The patterns are the following:

%speaker:     the name of the speaking character
%listener:    the name of the listening character
%speechid:    an ID number for the line; unique per package file
%searchtext:  the current contents of the "spoken text" text-box (except with
                underscores instead of spaces)
%tempspeech:  the file path of a temporary file from which the current speech
                audio data can be found
%tempgraph:   as above, but for DOT graph files (see below)

For example, if you have vlc player installed, you can use the following command
to play the selected line with it:
  start "" "c:/path/to/vlcplayer/vlc.exe" "%tempspeech" --play-and-exit

Other players probably work too, more or less. I used vlc because of its handy
--play-and-exit option. (Apparently there's some problem with either vlc or
windows here, as sometimes a couple of the first seconds after opening vlc are
inaudible.)

On Linux:
  mplayer -really-quiet "%tempspeech" &

You can also use these patterns with the output file text boxes.
For example: if you're searching and saving lots of sounds and don't want to
keep changing the text in the output file, you can enter something like this in
the output file box:
  c:/DeusExSpeeches/%speechid_%speaker_says_%searchtext_to_%listener.mp3

This will, when you click the user-defined command button, put the speech file
into c:/DeusExSpeeches/ directory, to a file with a somewhat descriptive name.

In the unlikely case that you actually want one of these patterns to appear
as-is in the command, you can put a backslash \ just before the % . To put an
actual backslash in the command, put two consecutive backslashes. (Note that
this has the unpleasant consequence that the backslashes so abundant in Windows
paths need to be duplicated, e.g. c:\foo\bar should be written as c:\\foo\\bar ,
which is why using forward slashes for paths here is more convenient.)


Conversation graph output
-------------------------

The program can output a graph representation of the conversation containing the
selected line. The graph is like a flow chart of the conversation, with
conversation events (speech, choices, item/credit transactions etc) represented
as boxes. Arrows between the boxes represent the ways the conversation can
proceed.

The ueconv program doesn't actually draw the graphs by itself; graphs are output
as DOT files. DOT which is a textual language for describing graphs. You'll need
Graphviz (a package of tools for turning DOT files into actual images) in order
to render them.

The basic way to output the graphs is with the text box
"Conversation graph DOT output file" and the button "Save conversation graph",
which work much like the speech output.

In addition, there's the "DOT command" text box and "Visualize conversation
graph" button. They're a way to conveniently view conversation graphs without
having to type the DOT commands yourself. Clicking the button will write the
conversation graph of the currently selected line to a file, and then
automatically invoke Graphviz in order to get a png from that. It will then
enter a mode where you can see the graph. In order to use this mode, the program
will need to know the correct command to invoke Graphviz's dot.exe program,
which must be written to the "DOT command" text box. That is, you need to have
Graphviz installed. Now, say it's installed to c:/graphviz ; the command should
then be something like c:/graphviz/release/bin/dot.exe .

In the visualization mode you can drag the image by holding the left mouse
button, and zoom with the mouse wheel. Also, the mode will detect speech nodes
in the graph, and clicking them with right mouse button will execute the
user-defined external command (see above) for that speech event (note, however,
that in this case the %searchtext parameter doesn't apply). With a properly set
user-defined command, this can be used to conveniently listen to the lines in
the conversation.


Default action for double-click in line list box
------------------------------------------------

With right mouse button, any but the "Find matches" button can be set as the
"default action", so that when you double-click on a line in the speech line
box, that action will be executed for the specific line. The default action
button is colored red; initially it is the user-defined command button.


Additional command line arguments
---------------------------------

The program can take some command line arguments in addition to the few
described earlier. Here are the most relevant ones. Note that there must not be
spaces around the equals sign =. As is usual with command line arguments, if the
argument itself (in this case the part after the equals sign) contains spaces,
it must be enclosed in quotes.

con-file-prefix=FOO       Set the initial contents of the
(or p=FOO)                "Package file name prefix" text box to FOO
(e.g. con-file-prefix="c:/some thing/DeusEx/System/DeusExCon"

spoken-text=FOO           Set the initial contents of the "Part of spoken text"
(or t=FOO)                text box to FOO.
(e.g. text="rotten way")

speaker=FOO               Set the initial contents of the "Part of speaker name"
(or a=FOO)                text box to FOO.
(e.g. speaker=jcd)

listener=FOO              Set the initial contents of the
(or b=FOO)                "Part of listener name" text box to FOO.
(e.g. listener=sandraren)

speech-out-file=FOO       Set the initial contents of the
(or s=FOO)                "Speech sound output file" text box to FOO.
(e.g. speech-out-file="c:/%speechid_%speaker_says_%searchtext_to_%listener.mp3")

graph-out-file=FOO       Set the initial contents of the
(or g=FOO)                "Conversation graph DOT output file" text box to FOO.
(e.g. graph-out-file="c:/%speechid_%speaker_says_%searchtext_to_%listener.dot")

user-command=FOO          Set the initial contents of the
(or u=FOO)                "User-defined external commands" text box to FOO.
(e.g. in Linux: user-command="mplayer \"%tempspeech\" &"
 or in Windows:
   u="start \"\" \"c:/path to vlc/vlc.exe\" \"%tempspeech\" --play-and-exit"
)

dot-command=FOO           Set the initial contents of the "DOT command" text box
(or d=FOO)                to FOO.
(e.g. dot-command=dot)

font-file=FOO             Use the font file FOO in the graphical interface.
(or f=FOO)                
(e.g. font-file=c:/windows/fonts/verdana.ttf)

font-size=FOO             Use font size FOO in the graphical interface.
(or F=FOO)
(e.g. font-size=12)

no-cache                  Disable package caching: only keep contents of one
(or nc)                   package file in memory at a time. If caching is
                          disabled, extracting audio repeatedly will be slower,
                          but less memory will be used. On modern computers,
                          however, memory is usually plenty, so this option
                          probably shouldn't be specified unless necessary.

A .bat file will probably be useful. See BATCH FILE section below.


There's also some additional functionality not described here, mostly related to
the textual interface.

----------
BATCH FILE
----------

It'll probably be the case that you don't want to re-type all of the command
line arguments every time you launch the program. For this a .bat (batch) file
is useful; you can use one to launch ueconv with specific parameters. There's
runme_example.bat in the binary_win directory that you can use as a starting
point; you'll need to at least modify the file paths (the .bat file can be
opened with e.g. Notepad). There are some details to know about Windows' command
line, such as escaping (backslash, and the double %'s) and line continuation
(the ^ characters), although simply modifying the paths shouldn't depend on
those very much.

------
ISSUES
------

The whole thing isn't very robust. For starters, error handling is very
rudimentary: often the program will just exit when an error happens, or, with
certain kinds of errors it might just do something weird. It may crash if given
an invalid package file. If the program abruptly exits due to an error, it'll
wait for a key press so that you can read possible error messages written to the
terminal (the command line window thingy).

Could be handy to be able to specify multiple package prefixes at once, for
games that use multiple sets of conversation files (e.g. The Nameless Mod).

Searching with speaker's and listener's in-game name instead of engine name
would be handy.

(And numerous others, probably.)

-------
CONTACT
-------

Please send bug reports, feature suggestions and what-not to
nuutti.holtta@luukku.com .

------
THANKS
------

Thanks for reading.

Thanks to people who wrote these:

http://wiki.beyondunreal.com/Legacy:Package_File_Format

"UT Package File Format document v1.6 (PDF)" at
  http://www.acordero.org/projects/
