Significant amount of roleplaying uses rules, by which I mean mechanics in the context of this post. Specifically mechanics made explicit by a book or note paper or oral agreements. In this post I will attempt to outline the scope of analysis that only considers the mechanics.
I will not analytically define mechanics here. (Reason: I don’t have a good necessary and sufficient conditions for something being a mechanic.) Let it suffice to say that anything manipulating numbers or ritual phrases or dots is a mechanic. The description of a trait or feat is a mechanic. Terms like scene or encounter may or may not be mechanics, depending on their level of codification.
A definition: A game is formal iff it has mechanics and if any event happening on the level of rules can be explained entirely by the rules.
The first condition is so that freeform, defined as being ruleless, does not count as formal.
The second condition means that if an observer can only see the rules-level exchange, that follower completely knows what is going on. Or: There is nothing outside the rules that can determine what will happen within the rules space.
I’ll call a game highly formal if it uses freeform or incompletely defined traits (e.g. my character is strong and so gets +3 to all rolls related to breaking things by brute force) and is otherwise formal. A game with such traits is not highly formal because the meaning of words in natural language is seldom exact and explicitly communicated.
Formal games exist: Chess. Highly formal games that are not formal exist: Universalis. Games that are neither formal nor highly formal exist: Freeform (and in the non-trivial case, say, Runequest). Hence, the definitions are meaningful.
The formality of roleplaying games
My claim is that there are very few games that are formal and roleplaying games. Highly formal roleplaying games are plentiful, though, and can be analysed in very similar way.
Almost all roleplaying games have a set of traits, player-defined or not, that have meaning according to the meaning of the relevant word in natural language. One possible exception I can name is called Rune. It is notable in that there is a strict by-the-point procedure for game master to use in constructing what amounts to dungeons; every trap, monster, and way out must be compensated for. It is precisely due to this extensive preparation that the game might succeed at being completely formal. All the qualifiers because it has been a while and I don’t remember if the game is actually completely formal.
My second claim was that highly formal games can be treated as formal without making significant errors. This is true because the ambiguity of highly formal games comes from the ambiguity of natural language. People who have played together for long, know each other, or are committed to certain source for their fiction (“this is a Star Wars game”) are likely to interpret the traits in similar way, or at least know the way in which the other players are likely to interpret the traits. Hence, the more the aforementioned factors are true, the close to formal the game is for that particular group. Treating a highly formal game as formal means simply going to the limit; it carries the assumption that the aforementioned conditions are sufficiently met.
Consider the set of rules of a particular game. Take a subset of this. The game this subset defines I hereby define a subsystem. (For more exact definition, consider that the structure of the rules-set or infastructure that supports it is not removed, only the rules directly defining player actions or consequences thereof are.)
Subsystems can be formal in much the same way as games can. For example, the character generation of several games is formal or highly formal. Combat systems may or may not be.
Any tool used for analysing formal games can be easily extended to the realm of (highly) formal roleplaying games, as well as select parts of other roleplaying games.
As an example let us consider game theory. For highly defined games one can simply assign utility to relevant mechanical outcomes and then consider the game as a standard one.