Changes: Extension:Cargo

Revision as of 22:29, 1 May 2020

For a complete Cargo user guide, see Extension:Cargo on mediawiki.org. This page does not attempt to duplicate information available there, and you should take advantage of both resources.

Cargo is an extension that lets pages on the wiki talk to each other. Imagine having a spreadsheet with data about your game that has a bunch of pages. One sheet talks about items. Another one talks about equipment. Yet another one tells you the stats of enemy bosses. Maybe there's one talking about different townspeople and their dialogue lines. You can imagine some columns that these might have - items have purchase costs, sell prices, ingredients, bonus armor. Enemy bosses have HP, defense, weaknesses, strengths, special skill thresholds. Townspeople have a greeting, a goodbye, an occupation, and a house. Cargo lets you actually create "virtual spreadsheets" in the form of "Cargo tables" that contain this information - and then makes them available so that any page on the wiki can use the information.

When you make wiki pages with infoboxes, you're already thinking about your data this way. You can see the similarities in the following questions:

Infoboxes	Spreadsheet Analogy	Cargo
What types of infoboxes do I need?	What Excel workbooks should I make?	What tables should I declare?
What fields go into this infobox?	What columns should I put into this excel sheet?	What fields should I include in my declaration?
What values should I put into this infobox / how should I format them?	What type of formatting do I apply to the cells in this column? (Date, decimal, percentage, etc)	What types should I assign to these fields? (DateTime, Boolean, String, etc)

Why Cargo?

The reason to use Cargo is so you can reuse the same data on multiple pages across your wiki. If you have items with stats, maybe you want to show the stats in the infobox on the item page, but also show them in a sortable table on the "Items" overview page. Or maybe you want to show a list of all dialogue lines of every person in one town on one page, and show the list of dialogue options of every blacksmith on another page - but you don't want to have to update both pages each time a new blacksmith from Whiterun is added in an expansion. Maybe your game has a crafting system, and you know that a dagger recipe is 2 iron ores. On another page, you want to show every item that requires at least one iron ore. Shouldn't there be some way that second page can just...know that Dagger should be there? Yep, it definitely is possible, with Cargo.

Vocabulary

Term	Spreadsheet Equivalent	Definition
Table	Excel sheet	A collection of similarly typed data objects along with their properties
Declare	Create new file	Tell the wiki about a new table you want it to make
Write to	Save	Save data to a table
Read from	Open	Get data from a table
Store		Add a new row and save
"Where" condition	Filtering a column	A condition that you require all returned fields to satisfy in order to be included in the result

How do I start?

Since Cargo is an extension, and not a default extension, the first step is to get Cargo on your wiki, which you can do by talking to your wiki manager. If you have development experience, it's highly recommended to use the Scribunto extension and write Lua modules as well - but if you don't want anything to do with coding, that's fine! You can still use Cargo without writing any line of code.

Once Cargo is enabled, there's three steps to creating a table.

Step 1: Declaration

In the Excel analogy, this is like opening a new workbook, creating column labels, and applying cell formatting to each column. That's it. There's no data stored yet, nothing is being retrieved, you are just telling the wiki what fields to expect when you do eventually store data.

A Cargo declaration might look something like this:

{{#cargo_declare:_table=WikiAbbreviations
|Platform=String<!-- Gamepedia or Fandom -->
|Abbreviation=String
|Full=String
|Definition=Text
}}

First we are giving the table a name: "WikiAbbreviations." Then we're saying, "Things in this table have 4 pieces of information we need to know: First, the platform, which will be either Gamepedia or Fandom (that's just a regular comment added as a note to editors). The next thing is the abbreviation we're talking about. Then we want to know what the abbreviation stands for, which we'll call "Full." Finally, what's the meaning of the abbreviation? We'll store that in a field called "Definition."

After you declare a table, you'll need to perform one more step before it's ready to be used - if you're an admin on your wiki, you can click a button that says "Create Data Table" on the template page after you've declared it and saved. If not, you can ask your Wiki Manager or another admin to help you. Every time you add or remove a field, or change the type of one of the fields, you'll have to recreate the table as well. This is done basically the same way - except now, instead of "Create" it'll say "Recreate." Again, you'll need to be an admin or ask for help from someone who's an admin on your wiki.

For additional information about recreating tables, see Recreating Tables.

Field Types

You'll want to see the documentation at MediaWiki.org for a full discussion of this. But here are a couple common field types you'll want to use:

String - default type
Wikitext string - same as string, but your wikitext formatting will be preserved in queries
Integer - a number that cannot have any fractional/decimal component
Float - a number that CAN have fractional/decimal component
Text - a large blob of text that you could NEVER EVER EVER possibly want to use in a "where" or "join" condition (more on that later!) - if in doubt, don't use this!
Wikitext - same thing as text, but your wikitext formatting will be preserved in queries.

Indexing

This section explains why I said you should avoid using Text & Wikitext in "where" or "join" conditions. If you want to just trust me on that, you can skip it.

Looking up data from Cargo tables is a slow process compared to some other things that you might do. In order to make it faster to find the rows that you're looking for when you provide a "where" condition (i.e. a filter), some fields are "indexed." But, indexing a field makes it slower to store in the first place. So Cargo gives you the option of using "text" and "wikitext" for very long fields. It's fine to query these to DISPLAY, but you should never FILTER based on their values, because that will be super slow. If you're confused about this, you can simply avoid the "Text" and "Wikitext" types. If you think you need them, ask your Wiki Manager for help!

Here's an example. Let's say we have a table of townspeople, and the field Greeting is Text type because some greetings can be very long. The other fields are all type String.

Okay:

|where= City="Whiterun"
|fields= Name, Occupation, Greeting

Not okay:

|where= Greeting="What do you get when you cross an arrow with a knee?"
|fields=Name, Occupation, City

Where do I declare a table?

You can do this in the <noinclude></noinclude> section of the template that you're storing the data from. Or you can do it on a standalone template that isn't used anywhere else. It's up to you - the only requirement is that it has to be done in the template namespace.

If the declaring template isn't the same as the template you're using to store, you'll need to use the parser function {{#cargo_attach:_table=TABLE_NAME}} in the noinclude section of your storing template. This tells the wiki to look for rows on the pages where your attaching template is used.

Step 2: Storing Data

Okay great so now we've basically opened a new workbook and added some column labels and types. What next? Well, it's pretty useless without some rows of data added to it. So let's add some data! Typically, you want to store data from an infobox template.

Here's the example store from our {{WikiAbbreviation}} example:

{{#cargo_store:_table=WikiAbbreviations
|Platform={{{platform|}}}
|Abbreviation={{{1|}}}
|Full={{{full|}}}
|Definition={{{definition}}}
}}

Seems easy enough...the user inputs |platform=, |full=, and |definition=, along with an unnamed first argument which is the Abbreviation. We just directly store all of this to the table without making any changes to what the user input.

Here's a slightly more complex example, from {{WikiDiscordLink}}:

{{#cargo_store:_table=WikiDiscords
|Invite={{{invite|}}}
|WikiName={{{name|}}}
|WikiURL={{{url|}}}
|Language={{#language:{{{lang|en}}}|en}}
|Platform={{#if:{{#pos:{{{url|}}}|.gamepedia.com}}|Gamepedia|Fandom}}
}}

Here we're storing data to a table called "WikiDiscords." This table has information about Discord servers belonging to various wikis. Here, we make the user input the URL of the wiki, and we store that. But, we also want to store a field called "Platform" which will be either "Fandom" or "Gamepedia." Your first thought might be, "well ok last time we made a |platform= field, why not do that again?" Well, because we're also asking for |url=, actually we can figure out what the platform is on our own from just that! If we find the string .gamepedia.com in |url= we're definitely on a Gamepedia wiki, and if not we're on a Fandom wiki. So let's just figure that out and then store Fandom or Gamepedia accordingly.

Why do we want the platform field if we are able to figure out platform from URL? Well, that brings us to our last stage...querying the data.

Step 3: Querying Data

So now we have a beautiful spreadsheet of data. It has a name, and it has a bunch of columns, with a bunch of rows that contain data. That's great and all, but...we haven't actually done anything yet. So let's do something!

"Doing something" means using the function {{#cargo_query:}}. Let's try making a table of ONLY Gamepedia abbreviations. Note that unlike declare, store, and attach, this parser function takes an argument of table and not _table. You can also use tables instead of table.

{{#cargo_query:table=WikiAbbreviations
|fields=Abbreviation,Full,Definition
|where=Platform="Gamepedia"
}}

No results

What if we wanted to show only Fandom abbreviations?

{{#cargo_query:table=WikiAbbreviations
|fields=Abbreviation,Full,Definition
|where=Platform="Fandom"
}}

No results

Turns out, it's pretty easy to do! Imagine for your wiki, if you were showing a list of items and all of their stats, or a list of townspeople and all their dialogue lines, or, or, or...It's pretty endless what you can do.

Advanced queries

See customizing tables for more information about adding markup to |format=table queries if you just want tables with markup.

So maybe you noticed that we didn't have much control over what the output looked like. All we could see was a table with all of the values. What if we want to just show a list of item icons to make a list of everything that one item builds into? What if we want to make a grid with every hero's thumbnail and a link to that hero's page, and display it on the wiki front page? Turns out, you can do all this and more - by setting |format=template (the default is |format=table when you don't set any |format= in your query).

Due to a bug with Cargo, actually using |format=template sometimes displays wrong. We have a workaround for this - using exactly the same syntax as |format=template queries, but {{CargoQuery}}^m instead of the actual #cargo_query parser function, you can get around this bug. If you need to use this, make sure Extension:Scribunto is enabled on your wiki. Talk to your Wiki Manager if unsure what to do. This workaround does not require you to use any Lua.

When you set |format=template, you can make a template that controls the format of every "row" in the output. So instead of being a table, you can display a list, or just a bunch of icons - whatever you want! Here are a few things you could choose to do:

Have the template output one row of a wikitable, but with more advanced styling in each row. This would require you to define both the table header and table end, using either |intro= and |outro= or just including extra elements surrounding your query.
Have the template display a thumbnail of one of the values along with its name, to make a navigation grid
Use #vardefine and #expr to calculate totals and then print them at the end (though also see Working with numbers)

See the MediaWiki Cargo documentation for more information.

Super-advanced stuff

Cargo tables can be joined to each other. The syntax is |join on=, or just join = in Lua.
In Lua, there is a dedicated function mw.ext.cargo.query, but if you want to declare or store from Lua, you need to use frame:callParserFunction. See also the Cargo Lua example.
Cargo tables are exposed via the MediaWiki api through action=cargoquery. This can be extremely useful.
If you want a table called [_pageData] set up, ask your Wiki Manager to have it built. See this wiki's table for an example of what fields you'll get.
The Path of Exile wiki and Leaguepedia both make extensive use of Cargo with Lua. Neither wiki should be emulated unless you are very comfortable coding, but if you want to see examples of what's possible you can look at these two wikis.

Working with numbers

Often you want Cargo to return numbers of things. Here is some advice.

SQL commands

You can use COUNT(*) to get the total number of entries that match your WHERE
You can use COUNT(DISTINCT FieldName) to get the total number of distinct entries in a field, for the rows that match your WHERE
SUM, MAX, MIN, etc exist. See Using SQL functions in the main Cargo docs for more information.

Using numbers from Cargo

One issue that people frequently encounter is that #cargo_query returns integers/floats with thousands separators. Wrap TRIM() around a field (e.g. |fields=TRIM(GoldCost)=GoldCost) to remove thousand separators if using #expr or other parser functions.
Another issue is that Cargo adds some extra HTML to outputs. If you want to do something like {{#expr:{{#cargo_query:QUERY HERE}} * 100 to display a percentage, you need to add the parameter |no html to your query. This will suppress the output of wrapping HTML and return just the number, so that #expr can understand it.
If you are using {{CargoQuery}}^m for your query, neither of the above problems should occur.

Miscellaneous common problems

HOLDS only works well when it's a single condition being queried. To get around this, use field__full LIKE/RLIKE or join the child table to the parent explicitly, e.g. Child._rowID=Parent._ID and then write out an = instead of a HOLDS.
As mentioned before, |format=template is buggy; use {{CargoQuery}}^m instead.
When using HOLDS, errors given in Lua are often completely incorrect from the actual issue. Ideally, just don't use HOLDS period. If you need to, ignore the error text if you get any.
Similar to the issue with thousands separators and numbers, you also may need to wrap fields that are stored via the {{PAGENAME}} magic word in a TRIM(). (_pageName is a default field but maybe you used {{#titleparts:}} for a field.)
Each template can only attach 1 table. So if you need a template to write to multiple tables, you might have to create "fake" templates that do nothing other than attach the table, but then are transcluded by the template doing the storing onto every page along with the store.
- Similarly, a template can only declare one table.
- A template CAN, however, both declare and attach - for a maximum of 2 tables available without any workarounds.
- See Attaching tables for more information.
unique is broken, don't use it

@@ Line 1: / Line 1: @@
-{{PAGENAME}} has been vandalized.{{Extension infobox
+{{Extension infobox
 | name = Cargo
 | version = 2.1.2