Source+Template,+A+Better+Free+Form

For a comprehensive comparison of the outputs from !MyFreeForm3, built-in and custom source templates, and Free Form see A Trio of Templates. =A Better Free Form Source Template= //Give me a template that is easy to use, produces good full and short footnotes without extraneous punctuation and exports cleanly via GEDCOM to other software!// toc
 * WARNING ** : This page has been extensively updated since 14 Oct 2011. !MyFreeForm3 supersedes !MyFreeForm and !MyFreeForm2 templates which are deprecated due to potential duplicating of the key term in the GEDCOM export along with an unwanted period.

RM4's Source Templates are a boon and frustrating at the same time. If you can choose the right one for a given source from some 417 different templates, use it correctly, you do end up with a footnote and bibliography style that is pretty consistent with what the gurus advise it should be. That's great for reports and websites generated from within RM4 but what gets exported to a standard GEDCOM for use by some other software is often pretty badly messed up.

The built-in Free Form template does a better job on exporting, you have total control over what gets into the TITL and PAGE tags, only limited by the order in which the tags are used to create footnotes. However, the Free Form template falls down in RM reports when a source is repeatedly cited, because there is only one Source Details field, //Page//, used by both the full, first footnote and by subsequent short footnotes. Thus, if lots of detail (e.g., think of a census citation) is entered in the Page field to ensure that the particular location within a source can readily be found, that same lengthy detail gets repeated in the subsequent short footnotes.

I have wrestled at length to come up with a Source Template that minimizes constraints (fewest fields=max flexibility), provides a short footnote if needed, exports nicely to standard GEDCOM, and has no extraneous punctuation. That has not been easy, given RM4's idiosyncrasies. I believe !MyFreeForm3 is about the best that one can do. This custom template mimics the built-in Free Form template, adding one field in Source Details: //Key: [ShortPage]//, for use in Short Footnotes instead of //Details:[Page]// (renamed from //Page:[Page]//). It also provides an additional field //Punctuation:[Punc]// in the Master Source with which you can override the default ", " separator in the footnote between master source and source details.

Import this Source Template
Download the template file to your computer. From RM4 main screen, select Lists > Source Templates to open the Source Templates window. Then select **Import** and browse to locate and select the .rmst file. If the new template was imported, at or near the top of the list of Source Types, you should see **!MyFreeForm3**. It's now available for the creation of new sources.

Guidance on entering data

 * 1) For //Footnote, Short Footnote and Bibliography//, follow the same practices as you would with the Free Form template, but do not end any with a period.
 * 2) The //Punctuation// field only affects footnotes in RM4 generated reports, not exports. Plug in a character to see the effect in the right pane.
 * 3) The two Source Details fields can be both filled, both null, or either one filled. Only when both are filled will the resulting footnote and short footnote differ in any way other than what you have defined in the corresponding Master Source fields. Filling neither or one is equivalent to the Free Form template.
 * 4) Ending footnotes with a period is pretty standard. Due to an export issue, the template must not automatically supply it. You provide the ending period at the end of the bottom-most Source Details field you use. If both Source Details fields are empty, the footnote will not end with a period (sorry, there's no way around it).
 * 5) On export, RM4 is going to stuff "; " between non-null //Details:[Page]// and //Key:[ShortPage]// which it puts into the PAGE tag. For consistency, that's what the full Footnote sentence template does, too.

Compose draft source with other templates
To mimic in !MyFreeForm3 the EE et al footnotes as well as one can, I try using a two-stage procedure. First, I'll use a Source Template suited to the type of source I am citing as a guide for how I may write a !MyFreeForm source. I fill in the fields of the selected template to generate the full and short footnotes and bibliography in the side panel, and copy these results to a word processor. Then I can parse the footnotes into their Master Source values (//Footnote// and //ShortFootnote//) and into their Source Details values (//Details:[Page]// and //Key:[ShortPage]//). Having done that, then I cancel the source creation and start over with the !MyFreeForm template, copying the values into the corresponding fields. That results in footnotes that resemble, to some degree, the gurus' footnotes, at least to the same level of detail, if not order.

Copy draft citations from Ancestry.com et al
Lots of sites provide citations of their resources and there are online Citation Machines. In effect, they have used their own templates. Breaking them down into component parts is the same as for the draft sources created using RM4 source templates, described above. It's a whole lot easier to parse them into !MyFreeForm3 than any other RM4 template, save the Free Form.

Examples
Have a look at this page for an example of a !MyFreeForm3 template used for the 1930 US Census: [|Source Template, A Better Free Form - Examples.pdf] and at this [|discussion] for an example of a directory as a source.

Merging
Merging of Free Form sources into !MyFreeForm3 sources is lossless, datawise. All the fields of Free Form are duplicated in !MyFreeForm3; only the //Punctuation:[Punc]// and //Key:[ShortPage]// fields are unique and will remain empty after the merge. However, to take advantage of the better Short Footnote, you will have to take out the key value from the //Details:[Page]// field and put it into the //Key:[ShortPage]//field of each citation. As long as //Key:[ShortPage]// remains empty, the Short Footnote will come out essentiallly the same as from Free Form.

=Global Conversion to !MyFreeForm3= Global conversion of Free Form sources to !MyFreeForm3 sources is easily and quickly done using SQLite. Open the database with your SQLite manager and inspect //SourceTemplateTable// for the value !MyFreeForm3 in the //Name// column. It will be at or near the bottom of the table with a //TemplateID// of 10000 or greater. Copy the following queries, edit them to plug in the TemplateID of !MyFreeForm3 in your database in place of //YourTemplateID// and run the queries to map all your FreeForm sources to the !MyFreeForm template: code format="sql" -- Map Free Form sources to !MyFreeForm3 source template UPDATE SourceTable SET TemplateID=YourTemplateID WHERE TemplateID=0 -- Add a [ShortPage] definition to every citation of a !MyFreeForm3 source lacking one -- else you will see the text "[ShortPage]" outputted in RM4 reports UPDATE CitationTable SET FIELDS = CAST(REPLACE(CAST(FIELDS AS TEXT),'', 'ShortPage') AS BLOB) WHERE CitationID IN  (SELECT CitationID FROM CitationTable C JOIN SourceTable S USING(SourceID)     WHERE TemplateID=YourTemplateID AND CAST(C.Fields AS TEXT) NOT LIKE '%ShortPage%'    ) -- A similar update of the SourceTable is required for the [Punc] field of !MyFreeForm3 -- else you will see the text "[Punc]" outputted in RM4 reports UPDATE SourceTable SET FIELDS = CAST(REPLACE(CAST(FIELDS AS TEXT),'ShortFootnote', 'PuncShortFootnote') AS BLOB) WHERE SourceID IN  (SELECT SourceID FROM SourceTable S     WHERE TemplateID=YourTemplateID AND CAST(S.FIELDS AS TEXT) NOT LIKE '%Punc%'    )

code Follow up to enter values in the //Key:[ShortPage]// field of each citation that needs an improved short footnote, and take out the duplicate values from the //Details:[Page]// field.

=How to find citations needing an improved Short Footnote= This section focuses on sources that have been globally converted to the !MyFreeForm3 templates.

Inspecting the Source List Report
For all source types and sources, one could create a RM4 Report > List > Source List including the Short Footnote and Citation Details (but not Detail Text, Comment and Media as that may clutter the report). Unfortunately, this report cannot be filtered by type of Source Template so you would find sources using the !MyFreeForm templates by searching for the term "[ShortPage]" in the report. It occurs in both the Footnote and the Short Footnote line. Doubly unfortunately, this report makes it hard to tell if a //Key:[ShortPage]// value has been entered, but you can use it to look for lengthy citations used more than once and thus likely to incur the use of the Short Footnote. Since the person and fact are identified in the Citation list for a Source in this report, you can then edit each citation accordingly. Best to fix one, __M__emorize it, __P__aste it into the other facts that use it, compare the original for differences in Detail Text, Comment and Media and resolve them on the copy, then delete the old. That guarantees that the key citation properties are truly identical so that Short Footnote and Ibid will come into play. This screenshot from the Source List report shows that both **Footnote**s contain the [ShortPage] field so this is a !MyFreeForm3 source. It has two identical citations for facts about the same person so will surely use either the Short Footnote or Ibid in reports. It also has citation or source detail values for both the //Details:[Page]// and the //Key:[ShortPage]// but it is hard to tell because they are merely separated by a semi-colon. Lengthier citations with embedded semi-colons are even tougher to deconstruct into the two fields. If there is no obvious key value repeated, then either Not overly helpful...
 * 1) [ShortPage] is null and the short footnote may be as verbose as the full one. The absence of any semi-colon "; " in the citation is absolute confirmation but the opposite is not necessarily true.
 * 2) This is a properly parsed !MyFreeForm3 citation. The absence of any semi-colon "; " in the citation is absolute refutation but the opposite is not necessarily true.

An SQLite query showing unparsed Source Details, longest first
Of course, SQLite could help in this process. I adapted an earlier query to produce the following report which shows the length (LenDet) of the field containing the [Page] value with a null value in [ShortPage]. Sorted in descending order, I work from the top down, re-running the query after each editing and replacement of duplicate citations (those with the same length most likely have identical values). Thus, the most verbose citations are tackled first. Some 156 characters out of the total shown under LenDet are used by XML tags, as can be seen at the bottom of the screen - that's the value of the highlighted cell which contains the values of all the fields in Source Details. Download