#include <cfl_tokenreader.h>

Classes
struct	LState

Public Types
enum	Mode { File , Stdin , String }

Public Member Functions
	TokenReader (Mode mode, const std::string &rInString="")

	TokenReader (const std::string &rFilename)

	~TokenReader (void)

void	Rewind (void)

std::istream *	Streamp (void)

Mode	SourceMode (void) const

std::string	FileName (void) const

bool	Peek (Token &token)

bool	Get (Token &token)

bool	ExistsBegin (const std::string &rLabel)

void	ReadBegin (const std::string &rLabel)

void	ReadBegin (const std::string &rLabel, Token &rToken)

void	ReadEnd (const std::string &rLabel)

void	SeekBegin (const std::string &rLabel)

void	SeekBegin (const std::string &rLabel, Token &rToken)

bool	Eos (const std::string &rLabel)

long int	ReadInteger (void)

double	ReadFloat (void)

std::string	ReadString (void)

std::string	ReadOption (void)

void	ReadBinary (std::string &rData)

void	ReadText (const std::string &rLabel, std::string &rText)

void	ReadVerbatim (const std::string &rLabel, std::string &rText)

void	ReadCharacterData (std::string &rData)

void	ReadSection (std::string &rSectionString)

bool	operator>> (Token &token)

int	Line (void) const

int	Level (void) const

bool	Recover (int level)

bool	Reset (int level=-1)

std::string	FileLine (void) const

Private Attributes
Mode	mMode

std::istream *	mpStream

std::ifstream	mFStream

std::istringstream *	mpSStream

std::string	mFileName

int	mLineCount

long int	mFilePos

bool	mFaudesComments

int	mLevel

std::vector< LState >	mLevelState

Token	mPeekToken

Detailed Description

A TokenReader reads sequential tokens from a file or string. It can get or peek the next token and it will track line numbers for informative diagnosis output.

The token stream is meant to be XML compliant, i.e., there are two dedicated token types that mark the begin and the end of XML elements while all other token types represent atomic data such as integers or strings that form the XML character data; see the documentation of the class Token for details.

The TokenReader maintains a current position in the stream and implements searching within nested elements and for sequential access of the tokens within each element; e.g. ReadBegin(const std::string&) will search for the element with the specified name.

Reading from the Tokenreader by a particular method encodes the type of the requested data, e.g. ReadInteger() to read an integer token. If the token at the current position does not match the requiested type, an exception is thrown. There are alose Get() and Peek() methods to retrieve and inspect the token at the current position.

For convenience, the TokenReader also implements reading all contents of an alement and larger chunks of data formtated as CDATA markup within an element; e.g. ReadVerbatim(const std::string&, std::string&) reads the contents of the specified element en block as one string.

Definition at line 64 of file cfl_tokenreader.h.

Member Enumeration Documentation

◆ Mode

enum faudes::TokenReader::Mode

Mode of operation: read from file, stdin or string

Enumerator
File
Stdin
String

Definition at line 70 of file cfl_tokenreader.h.

Constructor & Destructor Documentation

◆ TokenReader() [1/2]

faudes::TokenReader::TokenReader	(	Mode	mode,
		const std::string &	rInString = `""`
	)

TokenReader constructor

Parameters

mode	select source: File, Stdin or String
rInString	string to read from or filename, respectively.

Exceptions

Exception

faudes::Exception ios error opening file (id 1)
faudes::Exception invalid mode (Stdin not implemented) (id 1)

Definition at line 32 of file cfl_tokenreader.cpp.

◆ TokenReader() [2/2]

faudes::TokenReader::TokenReader ( const std::string & rFilename )

Creates a TokenReader for reading a file.

This is a convenience wrapper for TokenReader(Mode, const std::string&).

Parameters

rFilename file to read

Exceptions

Exception

faudes::Exception ios error opening file (id 1)

Definition at line 68 of file cfl_tokenreader.cpp.

◆ ~TokenReader()

faudes::TokenReader::~TokenReader ( void )

Destruct

Definition at line 89 of file cfl_tokenreader.cpp.

Member Function Documentation

◆ Eos()

bool faudes::TokenReader::Eos ( const std::string & rLabel )

Peek a token and check whether it ends the specified section. This function is meant for scanning a section with a while- construct.

ReadBegin("MySec"); 
while(!Eos("MySec")) { 
  ... 
}; 
ReadEnd("MySec");

Parameters

rLabel Token label to specify section

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
unexpected eof or section mismatch (id 51)

Returns: True at end of section

Definition at line 440 of file cfl_tokenreader.cpp.

◆ ExistsBegin()

bool faudes::TokenReader::ExistsBegin ( const std::string & rLabel )

Search for specified element

This function searches for the specified section on the current level. It skips any sections on the levels below, and it will wrap to the begin of the current section once. In the case of success, it returns true, else false. In contrast to other token i/o methodes, this method will not throw any execptions.

In the case of success, the next token is the begin-tag of the specified element, which can be read with ReadBegin().

Parameters

rLabel Token label to specify section

Returns: True if sectiob exists

Definition at line 315 of file cfl_tokenreader.cpp.

◆ FileLine()

std::string faudes::TokenReader::FileLine ( void ) const

Return "filename:line"

Definition at line 667 of file cfl_tokenreader.cpp.

◆ FileName()

std::string faudes::TokenReader::FileName ( void ) const

Access the filename.

Returns the name of the attached file, if any. For string mode and console mode dummy values are returned.

Returns: filename

Definition at line 125 of file cfl_tokenreader.cpp.

◆ Get()

bool faudes::TokenReader::Get ( Token & token )

Get next token.

Same as Peek() except that the token is removed from the buffer.

Parameters

token Reference to token

Exceptions

Exception faudes exception ios (id 1)

Returns: true for a valid token, false for eof or other stream errors

Definition at line 151 of file cfl_tokenreader.cpp.

◆ Level()

int faudes::TokenReader::Level ( void ) const

inline

Return current level of section nesting.

Returns: Number of lines read

Definition at line 494 of file cfl_tokenreader.h.

◆ Line()

int faudes::TokenReader::Line ( void ) const

Return number of lines read

Returns: Number of lines read

Definition at line 662 of file cfl_tokenreader.cpp.

◆ operator>>()

bool faudes::TokenReader::operator>> ( Token & token )

inline

Operator for get

Definition at line 476 of file cfl_tokenreader.h.

◆ Peek()

bool faudes::TokenReader::Peek ( Token & token )

Peek next token.

Copies the next token to the provided reference and returns true on success. The token remains in an internal buffer.

Technical note: we should have used a const-ref as return in orde to avoid the copy. However, this will require a tedious rewrite.

Parameters

token reference to token

Returns: true for a valid token, false for eof or other stream errors

Exceptions

Exception

faudes::Exception ios error reading file (id 1)

Definition at line 130 of file cfl_tokenreader.cpp.

◆ ReadBegin() [1/2]

void faudes::TokenReader::ReadBegin ( const std::string & rLabel )

Open a section by specified label. This function searches for the section on this level, it skips any sections on levels below this level, and it will wrap once to the begin of the current section. In the case of success, the matching begin token is the last token read. After processing the section, a matching ReadEnd(label) must be called. If the specified element does not exist, an exception is thrown.

Parameters

rLabel Token label to specify section

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Section begin label not found (id 51)

Definition at line 250 of file cfl_tokenreader.cpp.

◆ ReadBegin() [2/2]

void faudes::TokenReader::ReadBegin	(	const std::string &	rLabel,
		Token &	rToken
	)

Open a section by specified label.

This wrapper ReadBegin(const std::string&) will return the actual begin tag in its second argument, e.g., to inspect XML attributes.

Parameters

rLabel	Token label to specify section
rToken	Begin tag as found in token stream.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Section begin label not found (id 51)

Definition at line 256 of file cfl_tokenreader.cpp.

◆ ReadBinary()

void faudes::TokenReader::ReadBinary ( std::string & rData )

Read binary token.

Reads the next token and interprets it as an base64 encoded binary array.

Parameters

rData Buffer to read data

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Token mismatch (id 50)

Definition at line 503 of file cfl_tokenreader.cpp.

◆ ReadCharacterData()

void faudes::TokenReader::ReadCharacterData ( std::string & rData )

Read plain text

Read all text until and excluding the next markup tag. This method does no interpretation/substitution at all. It is meant to implemet carbon copy of text sections.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Stream mismatch (id 50)

Parameters

rData Buffer to read characterdata

Definition at line 610 of file cfl_tokenreader.cpp.

◆ ReadEnd()

void faudes::TokenReader::ReadEnd ( const std::string & rLabel )

Close the current section by matching the previous ReadBegin(). Reads all tokens up to and including end of current section.

Parameters

rLabel Token label to specify section

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Section end label not found (id 51)

Definition at line 365 of file cfl_tokenreader.cpp.

◆ ReadFloat()

double faudes::TokenReader::ReadFloat ( void )

Read float token

Reads the next token and interprets it as a float.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Token mismatch (id 50)

Returns: float value of token

Definition at line 463 of file cfl_tokenreader.cpp.

◆ ReadInteger()

long int faudes::TokenReader::ReadInteger ( void )

Read integer token

Reads the next token and interprets it as an non-negative integer.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Token mismatch (id 50)

Returns: value of integer token

Definition at line 450 of file cfl_tokenreader.cpp.

◆ ReadOption()

std::string faudes::TokenReader::ReadOption ( void )

Read option token

Reads the next token and interprets it as an option.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Token mismatch (id 50)

Returns: value of name token

Definition at line 490 of file cfl_tokenreader.cpp.

◆ ReadSection()

void faudes::TokenReader::ReadSection ( std::string & rSectionString )

Read XML section

Reads the current element, including all character data and markup, until and excluding the matching end tag. This method does no interpretation whatsoever. The result is a string that represents the respective section in plain XML format and can be used for expernal post-processing.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Stream mismatch (id 50)

Parameters

rSectionString Buffer to read section

Definition at line 634 of file cfl_tokenreader.cpp.

◆ ReadString()

std::string faudes::TokenReader::ReadString ( void )

Read string token

Reads the next token and interprets it as a string.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Token mismatch (id 50)

Returns: value of name token

Definition at line 476 of file cfl_tokenreader.cpp.

◆ ReadText()

void faudes::TokenReader::ReadText	(	const std::string &	rLabel,
		std::string &	rText
	)

Read plain text

Interpret the specified section as plain charater data section, read the character data and interpret relevant entities. Leading and trailing whitespaces are ignored, other formating is maintained.

This method facilitates the input of paragraphs of plain ASCII text with no other markup as the relevant entities (i.e. no HTML or RTF).

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Stream mismatch (id 50)

Parameters

rLabel	Buffer to read Text
rText	Name of section to read text from

Definition at line 517 of file cfl_tokenreader.cpp.

◆ ReadVerbatim()

void faudes::TokenReader::ReadVerbatim	(	const std::string &	rLabel,
		std::string &	rText
	)

Read verbatim text

Interpret the section as plain charater data section, read the character data from either one faudes string token or from consecutive CDATA markups. Leading and trailing whitespaces are ignored, other formating is maintained.

This method facilitates the input of paragraphs of plain ASCII text with excessive use of special characters, e.g., program fragments.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Stream mismatch (id 50)

Parameters

rLabel	Buffer to read Text
rText	Name of section to read text from

Definition at line 568 of file cfl_tokenreader.cpp.

◆ Recover()

bool faudes::TokenReader::Recover ( int level )

Recover by skipping tokens until returning to the specified level of section nesting

Returns: True on success

Definition at line 398 of file cfl_tokenreader.cpp.

◆ Reset()

bool faudes::TokenReader::Reset ( int level = -1 )

Reset to the begining of the specified level of section nesting.

Parameters

level target level, defaults to current level

Returns: True on success

Definition at line 413 of file cfl_tokenreader.cpp.

◆ Rewind()

void faudes::TokenReader::Rewind ( void )

Rewind stream

Reset the internal state to its initial value, i.e., the current position is the beginning of the stream. This is not functional in console mode.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)

Definition at line 99 of file cfl_tokenreader.cpp.

◆ SeekBegin() [1/2]

void faudes::TokenReader::SeekBegin ( const std::string & rLabel )

Find specified begin label.

This function searches for the section on this level and any descending level. It does not read the specified section tag, but stops just one token before (and in this regard matches the behaviour of ExistsBegin()).

Technical note: Former versions of libFAUDES also read the actual begin token and required a matching call of SeekEnd(). As of version 2.18a, this is not supported anymore. The previous behaviour was rarely needed and can be mimiqued by an ordinary ReadEnd() with a subsequent Recover(level).

Parameters

rLabel Label to specify section

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Section begin not found (id 51)

Definition at line 208 of file cfl_tokenreader.cpp.

◆ SeekBegin() [2/2]

void faudes::TokenReader::SeekBegin	(	const std::string &	rLabel,
		Token &	rToken
	)

Find specified begin label.

This version SeekBegin(const std::string&) will return the actual begin tag in its second argument.

Parameters

rLabel	Token label to specify section
rToken	Begin tag as found in token stream.

Exceptions

Exception

faudes::Exception ios error reading file (id 1)
Section begin label not found (id 51)

Definition at line 214 of file cfl_tokenreader.cpp.

◆ SourceMode()

Mode faudes::TokenReader::SourceMode ( void ) const

inline

Access stream mode.

Returns mode from construction, i.e. file, string or console.

Returns: Mode

Definition at line 137 of file cfl_tokenreader.h.

◆ Streamp()

std::istream * faudes::TokenReader::Streamp ( void )

Access C++ stream

This method provides direct access to the underlying C++ stream. After any such access, the TokenReader must be Rewind() to reset its internal state.

Returns: pointer to C++ input stream

Definition at line 94 of file cfl_tokenreader.cpp.

Member Data Documentation

◆ mFaudesComments

bool faudes::TokenReader::mFaudesComments

private

flag to ignore faudes comments marked by ''

Definition at line 543 of file cfl_tokenreader.h.

◆ mFileName

std::string faudes::TokenReader::mFileName

private

Filename

Definition at line 534 of file cfl_tokenreader.h.

◆ mFilePos

long int faudes::TokenReader::mFilePos

private

file position

Definition at line 540 of file cfl_tokenreader.h.

◆ mFStream

std::ifstream faudes::TokenReader::mFStream

private

actual stream object, file input

Definition at line 528 of file cfl_tokenreader.h.

◆ mLevel

int faudes::TokenReader::mLevel

private

Level (of nested sections)

Definition at line 546 of file cfl_tokenreader.h.

◆ mLevelState

std::vector<LState> faudes::TokenReader::mLevelState

private

Definition at line 556 of file cfl_tokenreader.h.

◆ mLineCount

int faudes::TokenReader::mLineCount

private

Line counter

Definition at line 537 of file cfl_tokenreader.h.

◆ mMode

Mode faudes::TokenReader::mMode

private

input mode

Definition at line 522 of file cfl_tokenreader.h.

◆ mPeekToken

Token faudes::TokenReader::mPeekToken

private

peek buffer

Definition at line 559 of file cfl_tokenreader.h.

◆ mpSStream

std::istringstream* faudes::TokenReader::mpSStream

private

actual stream object on heap, string input

Definition at line 531 of file cfl_tokenreader.h.

◆ mpStream

std::istream* faudes::TokenReader::mpStream

private

istream object pointer

Definition at line 525 of file cfl_tokenreader.h.

The documentation for this class was generated from the following files:

Classes

Public Types

Public Member Functions

Private Attributes

Detailed Description

Member Enumeration Documentation

◆ Mode

Constructor & Destructor Documentation

◆ TokenReader() [1/2]

◆ TokenReader() [2/2]

◆ ~TokenReader()

Member Function Documentation

◆ Eos()

◆ ExistsBegin()

◆ FileLine()

◆ FileName()

◆ Get()

◆ Level()

◆ Line()

◆ operator>>()

◆ Peek()

◆ ReadBegin() [1/2]

◆ ReadBegin() [2/2]

◆ ReadBinary()

◆ ReadCharacterData()

◆ ReadEnd()

◆ ReadFloat()

◆ ReadInteger()

◆ ReadOption()

◆ ReadSection()

◆ ReadString()

◆ ReadText()

◆ ReadVerbatim()

◆ Recover()

◆ Reset()

◆ Rewind()

◆ SeekBegin() [1/2]

◆ SeekBegin() [2/2]

◆ SourceMode()

◆ Streamp()

Member Data Documentation

◆ mFaudesComments

◆ mFileName

◆ mFilePos

◆ mFStream

◆ mLevel

◆ mLevelState

◆ mLineCount

◆ mMode

◆ mPeekToken

◆ mpSStream

◆ mpStream