Code::Blocks
SVN r11506
|
A parser threaded task, which can be assigned to the thread task pool, and run there. More...
#include <parserthread.h>
Public Member Functions | |
ParserThread (ParserBase *parent, const wxString &bufferOrFilename, bool isLocal, ParserThreadOptions &parserThreadOptions, TokenTree *tokenTree) | |
ParserThread constructor. More... | |
virtual | ~ParserThread () |
ParserThread destructor. More... | |
bool | Parse () |
Do the main job (syntax analysis) here. More... | |
bool | ParseBufferForNamespaces (const wxString &buffer, NameSpaceVec &result) |
Get the context "namespace XXX { ... }" directive. More... | |
bool | ParseBufferForUsingNamespace (const wxString &buffer, wxArrayString &result) |
Get the context "using namespace XXX" directive. More... | |
wxString | GetFilename () |
Public Member Functions inherited from cbThreadedTask | |
cbThreadedTask () | |
cbThreadedTask ctor More... | |
virtual | ~cbThreadedTask ()=0 |
cbThreadedTask dtor More... | |
void | Abort () |
This function is called to tell the task to abort (check cbThreadPool::AbortAllTasks) More... | |
Protected Types | |
enum | EClassType { ctStructure = 0, ctClass = 1, ctUnion = 3 } |
specify which "class like type" we are handling: struct or class or union More... | |
Protected Member Functions | |
int | Execute () |
Execute() is a virtual function derived from cbThreadedTask class, we should override it here. More... | |
wxChar | SkipToOneOfChars (const wxString &chars, bool supportNesting=false, bool singleCharToken=true) |
Continuously eat the tokens until we meet one of the matching characters. More... | |
void | DoParse () |
actually run the syntax analysis More... | |
void | SkipBlock () |
skip blocks {} More... | |
void | SkipAngleBraces () |
skip the template argument More... | |
void | HandleIncludes () |
handle include <XXXX> or include "XXXX" directive. More... | |
void | HandleNamespace () |
handle the statement: namespace XXX { More... | |
void | HandleClass (EClassType ct) |
handle class declaration More... | |
void | HandleFunction (wxString &name, bool isOperator=false, bool isPointer=false) |
handle function declaration or definition More... | |
void | HandleForLoopArguments () |
parse for loop arguments: for(int X; ... More... | |
void | HandleConditionalArguments () |
parse arguments like: if(int X = getNumber()) More... | |
void | HandleEnum () |
handle enum declaration More... | |
bool | CalcEnumExpression (Token *tokenParent, long &result, wxString &peek) |
calculate the value assigned to enumerator More... | |
void | HandleTypedef () |
handle typedef directive More... | |
bool | ReadVarNames () |
eg: class ClassA{...} varA, varB, varC This function will read the "varA, varB, varC" More... | |
bool | ReadClsNames (wxString &ancestor) |
handle class names, e.g., the code below More... | |
wxString | ReadAngleBrackets () |
read <> as a whole token More... | |
Token * | DoAddToken (TokenKind kind, const wxString &name, int line, int implLineStart=0, int implLineEnd=0, const wxString &args=wxEmptyString, bool isOperator=false, bool isImpl=false) |
add one token to the token tree More... | |
wxString | GetTokenBaseType () |
return the actual token's base type. More... | |
Protected Member Functions inherited from cbThreadedTask | |
bool | TestDestroy () const |
Be sure to call this function often. If it returns true, quit your task quickly. More... | |
bool | Aborted () const |
Same as TestDestroy() More... | |
Private Member Functions | |
bool | InitTokenizer () |
initialize the m_Buffer, load from local file or use a buffer in memory More... | |
Token * | TokenExists (const wxString &name, const Token *parent=0, short int kindMask=0xFFFF) |
if parent is 0, then global namespace will be used, all tokens under parent scope are searched More... | |
Token * | TokenExists (const wxString &name, const wxString &baseArgs, const Token *parent, TokenKind kind) |
Support function overloading. More... | |
Token * | FindTokenFromQueue (std::queue< wxString > &q, Token *parent=0, bool createIfNotExist=false, Token *parentIfCreated=0) |
TODO comment here? More... | |
bool | GetBaseArgs (const wxString &args, wxString &baseArgs) |
Converts a full argument list (including variable names) to argument types only and strips spaces. More... | |
void | GetTemplateArgs () |
Read the <xxxx=yyy, zzz> , and store the value in m_TemplateArgs. More... | |
void | ResolveTemplateArgs (Token *newToken) |
this function just associate the formal template argument to actual argument For example, we have such code: More... | |
wxArrayString | GetTemplateArgArray (const wxString &templateArgs, bool remove_gt_lt, bool add_last) |
normally the template argument is all in a wxString, this function just split them to a wxArrayString, each element is an actual argument. More... | |
void | SplitTemplateFormalParameters (const wxString &templateArgs, wxArrayString &formals) |
Split formal template argument list. More... | |
void | SplitTemplateActualParameters (const wxString &templateArgs, wxArrayString &actuals) |
Split actual template argument list. More... | |
bool | ResolveTemplateMap (const wxString &typeStr, const wxArrayString &actuals, std::map< wxString, wxString > &results) |
associate formal argument with actual template argument More... | |
void | RemoveTemplateArgs (const wxString &expr, wxString &expNoArgs, wxString &templateArgs) |
remove template arguments from an expression example: 'std::list<string>' will be separated into 'std::list' and '<string>' More... | |
bool | IsStillAlive (const wxString &funcInfo) |
Only for debug. More... | |
void | RefineAnonymousTypeToken (short int typeMask, wxString alise) |
change an anonymous(unnamed) token's name to a human readable name, the m_Str is expect to store the unnamed token name, for example, for parsing the code More... | |
Private Attributes | |
Tokenizer | m_Tokenizer |
if we regard the parserThread class as a syntax analyzer, then the Tokenizer class is regard as the lexer, which always feeds a wxString by calling m_Tokenizer.GetToken() More... | |
ParserBase * | m_Parent |
a pointer to its parent Parser object, the Parserthread class has two place to communicate with Parser class. More... | |
TokenTree * | m_TokenTree |
a pointer to the token tree, all the tokens will be added to that tree structure More... | |
Token * | m_LastParent |
parent Token, for example, when you are parsing in the class member variables, m_LastParent holds a pointer to the current context, which is a token holding class name More... | |
TokenScope | m_LastScope |
this member define the scope type of member variables, which is: public, private protected or undefined More... | |
wxString | m_Filename |
the file name of the parsing source More... | |
unsigned int | m_FileSize |
file size, actually the length of the wxString More... | |
unsigned int | m_FileIdx |
source file index on the "file map tree" More... | |
bool | m_IsLocal |
if true, means we are parsing a file which belongs to a C::B project, otherwise, we are parsing a file not belong to a C::B project(such as a system header file) More... | |
wxString | m_Str |
This is a very important member variables! It serves as a type stack, eg: parsing the statement: "unsigned int const varA;" we determine 'varA' is a token variable, until we searching to the last semicolon. More... | |
wxString | m_LastToken |
hold the previous token string More... | |
ParserThreadOptions | m_Options |
parser options, see the ParserThreadOptions structure More... | |
std::queue< wxString > | m_EncounteredNamespaces |
for member funcs implementation or a function declaration below eg: int ClassA::FunctionB(); EncounteredNamespaces is 'ClassA' More... | |
std::queue< wxString > | m_EncounteredTypeNamespaces |
namespaces in function return types for a function declaration below: e.g. More... | |
TokenIdxSet | m_UsedNamespacesIds |
globally included namespaces by "using namespace" statement More... | |
wxString | m_LastUnnamedTokenName |
TODO: describe me here. More... | |
bool | m_ParsingTypedef |
this makes a difference in unnamed class/struct/enum handling More... | |
wxString | m_Buffer |
a wxString holding the parsing buffer, if it is a file in the hard disk, then this stands for the file name. More... | |
wxString | m_PointerOrRef |
a pointer indicator or a references More... | |
wxString | m_TemplateArgument |
holds current template argument(s) when a template occurs More... | |
size_t | m_StructUnionUnnamedCount |
size_t | m_EnumUnnamedCount |
A parser threaded task, which can be assigned to the thread task pool, and run there.
This class represents a worker threaded task for the Code Completion plug-in, the main task is doing the syntax analysis and add every token to the token tree. The Token tree (sometimes, we call it TokenTree ) is a Patricia tree structure, more details can be seen in token.h and token.cpp. The buffer can either be loaded from a local file or directly used of a wxString.
Definition at line 138 of file parserthread.h.
|
protected |
specify which "class like type" we are handling: struct or class or union
Enumerator | |
---|---|
ctStructure | |
ctClass | |
ctUnion |
Definition at line 181 of file parserthread.h.
ParserThread::ParserThread | ( | ParserBase * | parent, |
const wxString & | bufferOrFilename, | ||
bool | isLocal, | ||
ParserThreadOptions & | parserThreadOptions, | ||
TokenTree * | tokenTree | ||
) |
ParserThread constructor.
parent | the parent Parser object which contain the token tree. |
bufferOrFilename | it's either the filename to open or a wxString buffer already in memory. |
isLocal | determine whether this is a file locally belong to a cbp or in other global paths. |
parserThreadOptions | parser thread options, see ParserThreadOptions Class for details. |
tokenTree | it is the tree structure holding all the tokens, ParserThread will add every token when it parsed. |
Definition at line 197 of file parserthread.cpp.
References _T, cbThrow, m_Tokenizer, m_TokenTree, Tokenizer::SetTokenizerOption(), ParserThreadOptions::storeDocumentation, and ParserThreadOptions::wantPreprocessor.
|
virtual |
ParserThread destructor.
Definition at line 222 of file parserthread.cpp.
References ParserThreadOptions::loader, m_Options, and LoaderBase::Sync().
|
protected |
calculate the value assigned to enumerator
Definition at line 2795 of file parserthread.cpp.
References _T, Expression::AddToInfixExpression(), TokenTree::at(), Expression::CalcPostfix(), ParserConsts::clbrace, Expression::Clear(), ParserConsts::comma, ParserConsts::commaclbrace, Expression::ConvertInfixToPostfix(), ParserConsts::dcolon, wxString::Format(), Expression::GetResult(), Tokenizer::GetState(), Expression::GetStatus(), Tokenizer::GetToken(), IS_ALIVE, wxString::IsEmpty(), Token::m_Args, Token::m_Index, m_Tokenizer, m_TokenTree, Tokenizer::SetState(), SkipToOneOfChars(), wxString::StartsWith(), tkEnumerator, TokenTree::TokenExists(), wxString::ToLong(), tsRawExpression, ParserConsts::underscore_chr(), Tokenizer::UngetToken(), wxIsalpha(), and wxIsdigit().
Referenced by HandleEnum().
|
protected |
add one token to the token tree
kind | Token type, see Emun for more details |
name | Token name, this is the key string to be searched in the token tree |
line | line number of the source file where the current Token locates |
implLineStart | if this is a function implementation, it is the start of function body |
implLineEnd | like the one above, it is the end line of the function implementation body |
args | if the token type is a function, this is the function arguments |
isOperator | bool variable to determine an operator override function or not |
isTmpl | bool variable to determine it is a function declaration or implementation |
Definition at line 1475 of file parserthread.cpp.
References _T, Token::AddChild(), ADDTOKEN, TokenTree::at(), wxString::Clear(), ParserConsts::dcolon, wxString::Find(), FindTokenFromQueue(), GetBaseArgs(), GetTokenBaseType(), Token::GetTokenKindString(), TokenTree::insert(), TokenTree::InsertTokenBelongToFile(), wxString::IsEmpty(), ParserThreadOptions::isTemp, Token::m_Args, Token::m_BaseArgs, Token::m_BaseType, m_EncounteredNamespaces, m_EncounteredTypeNamespaces, Token::m_FileIdx, m_FileIdx, Token::m_FullType, Token::m_ImplFileIdx, Token::m_ImplLine, Token::m_ImplLineEnd, Token::m_ImplLineStart, Token::m_Index, Token::m_IsLocal, m_IsLocal, Token::m_IsOperator, Token::m_IsTemp, m_LastParent, m_LastScope, Token::m_Line, Token::m_Name, m_Options, Token::m_ParentIndex, m_PointerOrRef, Token::m_Scope, m_Str, Token::m_TemplateArgument, m_TemplateArgument, m_Tokenizer, Token::m_TokenKind, TokenTree::m_TokenTicketCount, m_TokenTree, ParserThreadOptions::parentIdxOfBuffer, wxString::Prepend(), Tokenizer::SetLastTokenIdx(), ParserConsts::space_chr(), ParserConsts::tilde, tkAnyContainer, tkAnyFunction, tkClass, tkConstructor, tkDestructor, tkVariable, TokenExists(), TRACE, wxString::Trim(), wxString::wx_str(), wxEmptyString, and wxNOT_FOUND.
Referenced by DoParse(), HandleClass(), HandleConditionalArguments(), HandleEnum(), HandleForLoopArguments(), HandleFunction(), HandleNamespace(), HandleTypedef(), ReadClsNames(), and ReadVarNames().
|
protected |
actually run the syntax analysis
Definition at line 545 of file parserthread.cpp.
References _T, ParserThreadOptions::bufferSkipBlocks, ParserConsts::clarray, ParserConsts::clbrace_chr(), wxString::Clear(), wxString::clear(), ParserConsts::colon, ParserConsts::colon_chr(), ParserConsts::comma, ParserConsts::comma_chr(), ParserConsts::commasemicolonopbrace, ctClass, ctStructure, ctUnion, ParserConsts::dash, ParserConsts::dash_chr(), ParserConsts::dcolon, DoAddToken(), ParserConsts::dot_chr(), ParserConsts::equals, ParserConsts::equals_chr(), wxString::find(), FindTokenFromQueue(), wxString::GetChar(), Tokenizer::GetLineNumber(), wxStringTokenizer::GetNextToken(), Tokenizer::GetState(), GetTemplateArgs(), Tokenizer::GetToken(), ParserConsts::gt_chr(), HandleClass(), ParserThreadOptions::handleClasses, HandleConditionalArguments(), HandleEnum(), ParserThreadOptions::handleEnums, HandleForLoopArguments(), HandleFunction(), ParserThreadOptions::handleFunctions, HandleIncludes(), HandleNamespace(), HandleTypedef(), ParserThreadOptions::handleTypedefs, ParserThreadOptions::handleVars, ParserConsts::hash_chr(), wxStringTokenizer::HasMoreTokens(), IS_ALIVE, wxString::IsEmpty(), wxString::IsSameAs(), Token::IsValidAncestor(), ParserConsts::kw___asm, ParserConsts::kw___at, ParserConsts::kw__C_, ParserConsts::kw__CPP_, ParserConsts::kw_attribute, ParserConsts::kw_case, ParserConsts::kw_catch, ParserConsts::kw_class, ParserConsts::kw_const, ParserConsts::kw_declspec, ParserConsts::kw_delete, ParserConsts::kw_do, ParserConsts::kw_else, ParserConsts::kw_enum, ParserConsts::kw_extern, ParserConsts::kw_for, ParserConsts::kw_friend, ParserConsts::kw_if, ParserConsts::kw_include, ParserConsts::kw_inline, ParserConsts::kw_namespace, ParserConsts::kw_noexcept, ParserConsts::kw_operator, ParserConsts::kw_private, ParserConsts::kw_protected, ParserConsts::kw_public, ParserConsts::kw_return, ParserConsts::kw_static, ParserConsts::kw_struct, ParserConsts::kw_switch, ParserConsts::kw_template, ParserConsts::kw_typedef, ParserConsts::kw_union, ParserConsts::kw_using, ParserConsts::kw_virtual, ParserConsts::kw_volatile, ParserConsts::kw_while, wxString::Length(), ParserConsts::lt, Token::m_AncestorsString, Token::m_BaseType, m_EncounteredNamespaces, m_EncounteredTypeNamespaces, Token::m_FullType, Token::m_Index, m_LastParent, m_LastScope, m_LastToken, m_LastUnnamedTokenName, Token::m_Name, m_Options, m_PointerOrRef, m_Str, m_TemplateArgument, m_Tokenizer, Token::m_TokenKind, m_UsedNamespacesIds, Tokenizer::NotEOF(), ParserConsts::oparray_chr(), ParserConsts::opbrace, ParserConsts::opbrace_chr(), ParserConsts::opbracket_chr(), Tokenizer::PeekToken(), ParserConsts::plus_chr(), ParserConsts::ptr, ParserConsts::ptr_chr(), ParserConsts::question_chr(), ReadAngleBrackets(), ParserConsts::ref_chr(), ResolveTemplateArgs(), ParserConsts::semicolon, ParserConsts::semicolon_chr(), ParserConsts::semicolonclbrace, ParserConsts::semicolonopbrace, Tokenizer::SetState(), SkipAngleBraces(), SkipBlock(), Tokenizer::SkipToEOL(), SkipToOneOfChars(), ParserConsts::space_chr(), wxString::StartsWith(), tkNamespace, tkTypedef, tkVariable, TRACE, wxString::Trim(), tsNormal, tsPrivate, tsProtected, tsPublic, tsUndefined, ParserConsts::underscore_chr(), Tokenizer::UngetToken(), ParserThreadOptions::useBuffer, wxString::wx_str(), wxIsalpha(), and wxNOT_FOUND.
Referenced by HandleClass(), HandleNamespace(), and Parse().
|
inlineprotectedvirtual |
Execute() is a virtual function derived from cbThreadedTask class, we should override it here.
In the batch parsing mode, a lot of parser threads were generated and executed concurrently, this often happens when user open a project. Every parserthread task will firstly be added to the thread pool, later called automatically from the thread pool.
Implements cbThreadedTask.
Definition at line 188 of file parserthread.h.
References CC_LOCKER_TRACK_TT_MTX_LOCK, CC_LOCKER_TRACK_TT_MTX_UNLOCK, s_TokenTreeMutex, and wxEmptyString.
|
private |
TODO comment here?
Definition at line 1432 of file parserthread.cpp.
References _T, Token::AddChild(), TokenTree::insert(), m_FileIdx, Token::m_Index, Token::m_IsLocal, m_IsLocal, Token::m_Name, Token::m_ParentIndex, Token::m_TokenKind, TokenTree::m_TokenTicketCount, m_TokenTree, tkClass, tkNamespace, TokenExists(), TRACE, and wxString::wx_str().
Referenced by DoAddToken(), DoParse(), and HandleFunction().
Converts a full argument list (including variable names) to argument types only and strips spaces.
eg: if the argument list is like "(const TheClass* the_class = 0x1234, int my_int = 567)" then, the returned argument list is "(const TheClass*,int)"
args | Full argument list |
baseArgs | argument types only |
Definition at line 3231 of file parserthread.cpp.
References _T, wxString::Alloc(), ParserConsts::clarray_chr(), ParserConsts::clbracket_chr(), ParserConsts::colon_chr(), ParserConsts::comma_chr(), ParserConsts::eol_chr(), wxString::Find(), ParserConsts::gt_chr(), ParserConsts::kw_attribute, ParserConsts::kw_const, ParserConsts::kw_enum, ParserConsts::kw_volatile, wxString::Len(), ParserConsts::lt_chr(), wxString::Mid(), ParserConsts::null(), ParserConsts::oparray_chr(), ParserConsts::opbracket_chr(), ParserConsts::ptr, ParserConsts::ptr_chr(), ParserConsts::ref_chr(), ParserConsts::space_chr(), TRACE, ParserConsts::underscore_chr(), wxString::wx_str(), wxIsalnum(), and wxNOT_FOUND.
Referenced by DoAddToken().
|
inline |
Definition at line 177 of file parserthread.h.
|
private |
normally the template argument is all in a wxString, this function just split them to a wxArrayString, each element is an actual argument.
Definition at line 3534 of file parserthread.cpp.
References wxArrayString::Add(), wxString::clear(), ParserConsts::comma_chr(), wxString::GetChar(), ParserConsts::gt_chr(), wxString::IsEmpty(), wxString::Len(), ParserConsts::lt_chr(), wxString::Remove(), wxString::RemoveLast(), ParserConsts::space_chr(), and wxString::Trim().
Referenced by SplitTemplateActualParameters(), and SplitTemplateFormalParameters().
|
private |
Read the <xxxx=yyy, zzz> , and store the value in m_TemplateArgs.
Definition at line 3473 of file parserthread.cpp.
References wxString::clear(), Tokenizer::GetState(), Tokenizer::GetToken(), ParserConsts::gt, IS_ALIVE, wxString::IsEmpty(), ParserConsts::lt, m_TemplateArgument, m_Tokenizer, ParserConsts::semicolon, Tokenizer::SetState(), tsNormal, and Tokenizer::UngetToken().
Referenced by DoParse(), HandleClass(), and HandleTypedef().
|
protected |
return the actual token's base type.
e.g.: if the token type string is: "const wxString &" then, the actual token base type is : "wxString"
Definition at line 1351 of file parserthread.cpp.
References _T, ParserConsts::colon_chr(), wxString::GetChar(), wxString::IsSameAs(), ParserConsts::kw_const, wxString::Length(), m_Str, wxString::Mid(), ParserConsts::ptr_chr(), ParserConsts::ref_chr(), wxString::Remove(), TRACE, ParserConsts::underscore_chr(), wxString::wx_str(), wxIsalnum(), and wxIsspace().
Referenced by DoAddToken().
|
protected |
handle class declaration
ct | specify class like type : struct or enum or class |
Definition at line 1882 of file parserthread.cpp.
References _T, wxString::Clear(), ParserConsts::colon, ParserConsts::comma, ParserConsts::comma_chr(), ctClass, ctUnion, ParserConsts::dcolon, DoAddToken(), DoParse(), ParserConsts::equals, wxString::GetChar(), wxArrayString::GetCount(), Tokenizer::GetLineNumber(), wxStringTokenizer::GetNextToken(), Tokenizer::GetState(), GetStringFromArray(), GetTemplateArgs(), Tokenizer::GetToken(), ParserConsts::gt, HandleFunction(), ParserThreadOptions::handleVars, wxStringTokenizer::HasMoreTokens(), IS_ALIVE, wxString::IsEmpty(), wxArrayString::IsEmpty(), ParserConsts::kw_attribute, ParserConsts::kw_declspec, ParserConsts::kw_private, ParserConsts::kw_protected, ParserConsts::kw_public, ParserConsts::lt, Token::m_AncestorsString, m_FileIdx, Token::m_ImplLine, Token::m_ImplLineEnd, Token::m_ImplLineStart, Token::m_IsAnonymous, m_LastParent, m_LastScope, m_LastUnnamedTokenName, Token::m_Name, m_Options, m_ParsingTypedef, m_PointerOrRef, m_Str, m_StructUnionUnnamedCount, Token::m_TemplateArgument, m_TemplateArgument, Token::m_TemplateType, m_Tokenizer, Token::m_TokenKind, ParserConsts::opbrace, ParserConsts::opbracket_chr(), Tokenizer::PeekToken(), wxString::Prepend(), wxString::Printf(), ParserConsts::ptr_chr(), ReadClsNames(), ReadVarNames(), ParserConsts::ref_chr(), ParserConsts::semicolon, Tokenizer::SetState(), SkipAngleBraces(), SkipToOneOfChars(), ParserConsts::space, ParserConsts::space_chr(), SplitTemplateFormalParameters(), tkClass, tkVariable, TokenExists(), TRACE, tsNormal, tsPrivate, tsPublic, Tokenizer::UngetToken(), and wxString::wx_str().
Referenced by DoParse(), and HandleTypedef().
|
protected |
parse arguments like: if(int X = getNumber())
Definition at line 2452 of file parserthread.cpp.
References _T, wxString::clear(), DoAddToken(), wxString::empty(), wxString::EndsWith(), Tokenizer::GetFilename(), Tokenizer::GetLineNumber(), Tokenizer::GetToken(), Tokenizer::InitFromBuffer(), IS_ALIVE, wxString::IsEmpty(), wxString::length(), m_PointerOrRef, m_Str, m_TemplateArgument, m_Tokenizer, wxString::Mid(), Tokenizer::PeekToken(), ParserConsts::ptr_chr(), ParserConsts::ref_chr(), RemoveTemplateArgs(), ResolveTemplateArgs(), wxString::StartsWith(), tkVariable, and TRACE.
Referenced by DoParse().
|
protected |
handle enum declaration
Definition at line 2645 of file parserthread.cpp.
References _T, CalcEnumExpression(), ParserConsts::clbrace, wxString::Clear(), ParserConsts::colon, ParserConsts::comma, ParserConsts::commaclbrace, DoAddToken(), ParserConsts::equals, g_UnnamedSymbol, wxString::GetChar(), Tokenizer::GetLineNumber(), Tokenizer::GetNestingLevel(), Tokenizer::GetState(), Tokenizer::GetToken(), ParserThreadOptions::handleEnums, IS_ALIVE, wxString::IsEmpty(), ParserConsts::kw_class, m_EnumUnnamedCount, m_FileIdx, Token::m_ImplLine, Token::m_ImplLineEnd, Token::m_ImplLineStart, Token::m_IsAnonymous, m_LastParent, m_LastUnnamedTokenName, m_Options, m_ParsingTypedef, Token::m_Scope, m_Str, m_Tokenizer, ParserConsts::opbrace, ParserConsts::opbrace_chr(), Tokenizer::PeekToken(), wxString::Printf(), ParserConsts::semicolon, Tokenizer::SetState(), SkipToOneOfChars(), tkEnum, tkEnumerator, tkVariable, TokenExists(), tsNormal, tsPrivate, tsPublic, ParserConsts::underscore_chr(), Tokenizer::UngetToken(), wxString::wx_str(), and wxIsalpha().
Referenced by DoParse(), and HandleTypedef().
|
protected |
parse for loop arguments: for(int X; ...
; ...)
Definition at line 2532 of file parserthread.cpp.
References _T, wxString::clear(), ParserConsts::colon, ParserConsts::comma, DoAddToken(), wxString::empty(), wxString::EndsWith(), ParserConsts::equals, Tokenizer::GetFilename(), Tokenizer::GetLineNumber(), Tokenizer::GetToken(), Tokenizer::InitFromBuffer(), IS_ALIVE, wxString::IsEmpty(), wxString::length(), m_PointerOrRef, m_Str, m_TemplateArgument, m_Tokenizer, wxString::Mid(), Tokenizer::PeekToken(), ParserConsts::ptr_chr(), ParserConsts::ref_chr(), RemoveTemplateArgs(), ResolveTemplateArgs(), ParserConsts::semicolon, wxString::StartsWith(), tkVariable, and TRACE.
Referenced by DoParse().
|
protected |
handle function declaration or definition
name | function name |
isOperator | if true, means it is an operator overload function |
isPointer | if true, means it is a function pointer |
Definition at line 2257 of file parserthread.cpp.
References _T, ParserConsts::clbrace, wxString::Clear(), ParserConsts::colon, ParserConsts::comma, DoAddToken(), ParserConsts::equals, wxString::find(), FindTokenFromQueue(), Tokenizer::GetLineNumber(), Tokenizer::GetToken(), wxString::IsEmpty(), ParserConsts::kw_catch, ParserConsts::kw_const, ParserConsts::kw_friend, ParserConsts::kw_noexcept, ParserConsts::kw_throw, ParserConsts::kw_try, m_EncounteredNamespaces, m_EncounteredTypeNamespaces, m_Filename, Token::m_IsConst, Token::m_IsNoExcept, m_LastParent, Token::m_Name, m_Options, m_Str, Token::m_TemplateArgument, m_TemplateArgument, Token::m_TemplateMap, m_Tokenizer, ParserConsts::oparray_chr(), ParserConsts::opbrace, Tokenizer::PeekToken(), ParserConsts::ptr, wxString::Remove(), wxString::RemoveLast(), ResolveTemplateArgs(), ParserConsts::semicolon, SkipBlock(), SkipToOneOfChars(), wxString::StartsWith(), ParserConsts::tilde, tkConstructor, tkDestructor, tkFunction, TRACE, wxString::Trim(), Tokenizer::UngetToken(), ParserThreadOptions::useBuffer, wxString::wx_str(), wxNOT_FOUND, and wxT.
Referenced by DoParse(), and HandleClass().
|
protected |
handle include <XXXX> or include "XXXX" directive.
This will internally add another parserThead object associate with the included file to parserthread pool
Definition at line 1662 of file parserthread.cpp.
References _T, ParserCommon::FileType(), ParserThreadOptions::followGlobalIncludes, ParserThreadOptions::followLocalIncludes, ParserCommon::ftOther, wxString::GetChar(), ParserBase::GetFullFileName(), Tokenizer::GetToken(), ParserConsts::gt_chr(), IS_ALIVE, wxString::IsEmpty(), TokenTree::IsFileParsed(), wxString::Length(), ParserConsts::lt_chr(), m_Filename, m_IsLocal, m_Options, m_Parent, m_Tokenizer, m_TokenTree, ParserBase::ParseFile(), TRACE, and wxString::wx_str().
Referenced by DoParse().
|
protected |
handle the statement: namespace XXX {
Definition at line 1744 of file parserthread.cpp.
References _T, wxArrayString::Add(), ParserConsts::dcolon, DoAddToken(), DoParse(), ParserConsts::equals, Tokenizer::GetLineNumber(), Tokenizer::GetToken(), IS_ALIVE, Token::m_Aliases, m_FileIdx, Token::m_ImplFileIdx, Token::m_ImplLine, Token::m_ImplLineEnd, Token::m_ImplLineStart, m_LastParent, m_LastScope, m_Tokenizer, NULL, ParserConsts::opbrace, Tokenizer::PeekToken(), ParserConsts::semicolonopbrace, Tokenizer::SetState(), SkipToOneOfChars(), tkNamespace, TokenExists(), TRACE, tsNormal, tsPublic, and Tokenizer::UngetToken().
Referenced by DoParse().
|
protected |
handle typedef directive
Definition at line 2874 of file parserthread.cpp.
References _(), _T, wxString::BeforeFirst(), ParserConsts::clbracket_chr(), wxString::Clear(), ParserConsts::comma, ctClass, ctStructure, ctUnion, ParserConsts::dcolon, DoAddToken(), wxString::Find(), wxString::GetChar(), Tokenizer::GetLineNumber(), GetTemplateArgs(), Tokenizer::GetToken(), HandleClass(), HandleEnum(), wxArrayString::Index(), IS_ALIVE, wxString::IsEmpty(), wxArrayString::IsEmpty(), ParserConsts::kw_class, ParserConsts::kw_const, ParserConsts::kw_enum, ParserConsts::kw_struct, ParserConsts::kw_union, ParserConsts::lt, ParserConsts::lt_chr(), m_LastParent, m_LastUnnamedTokenName, m_ParsingTypedef, m_PointerOrRef, m_Str, m_TemplateArgument, Token::m_TemplateType, m_Tokenizer, Token::m_TokenKind, wxString::Mid(), ParserConsts::opbracket_chr(), Tokenizer::PeekToken(), ParserConsts::ptr, ParserConsts::ptr_chr(), ReadClsNames(), ParserConsts::ref, wxString::Remove(), wxString::RemoveLast(), ResolveTemplateArgs(), ParserConsts::semicolon, ParserConsts::space_chr(), tkClass, tkTypedef, TRACE, wxString::Trim(), Tokenizer::UngetToken(), wxString::wx_str(), and wxNOT_FOUND.
Referenced by DoParse().
|
private |
initialize the m_Buffer, load from local file or use a buffer in memory
Definition at line 468 of file parserthread.cpp.
References _T, Delete(), ParserThreadOptions::fileOfBuffer, Tokenizer::Init(), Tokenizer::InitFromBuffer(), ParserThreadOptions::initLineOfBuffer, TokenTree::InsertFileOrGetIndex(), wxString::IsEmpty(), wxFile::IsOpened(), wxFile::Length(), ParserThreadOptions::loader, m_Buffer, m_FileIdx, m_Filename, m_FileSize, m_Options, m_Tokenizer, m_TokenTree, TRACE, ParserThreadOptions::useBuffer, wxString::wx_str(), and wxFileExists().
Referenced by Parse().
|
private |
Only for debug.
Definition at line 3740 of file parserthread.cpp.
References _T, cbThreadedTask::TestDestroy(), and TRACE.
bool ParserThread::Parse | ( | ) |
Do the main job (syntax analysis) here.
Definition at line 510 of file parserthread.cpp.
References _T, DoParse(), TokenTree::FlagFileAsParsed(), InitTokenizer(), IS_ALIVE, Tokenizer::IsOK(), m_FileIdx, m_Filename, m_Options, m_ParsingTypedef, m_Tokenizer, m_TokenTree, TokenTree::ReserveFileForParsing(), TRACE, ParserThreadOptions::useBuffer, and wxString::wx_str().
Referenced by Parser::Parse(), ParserBase::ParseBuffer(), Parser::ParseBuffer(), Parser::ParseBufferForFunctions(), and ParserBase::Reparse().
bool ParserThread::ParseBufferForNamespaces | ( | const wxString & | buffer, |
NameSpaceVec & | result | ||
) |
Get the context "namespace XXX { ... }" directive.
It is used to find the initial search scope before CC prompt a suggestion list. Need a critical section locker before call this function!
buffer | wxString to be parsed. |
result | vector containing all the namespace names. |
Definition at line 321 of file parserthread.cpp.
References wxArrayString::Add(), ParserConsts::clbrace, ParserConsts::dcolon, NameSpace::EndLine, ParserConsts::equals, wxArrayString::GetCount(), Tokenizer::GetLineNumber(), Tokenizer::GetToken(), Tokenizer::InitFromBuffer(), IS_ALIVE, wxString::IsEmpty(), wxArrayString::IsEmpty(), Tokenizer::IsOK(), ParserConsts::kw_namespace, ParserConsts::kw_using, m_ParsingTypedef, m_Tokenizer, NameSpace::Name, Tokenizer::NotEOF(), ParserConsts::opbrace, Tokenizer::PeekToken(), wxArrayString::RemoveAt(), ParserConsts::semicolonclbrace, Tokenizer::SetState(), SkipBlock(), SkipToOneOfChars(), NameSpace::StartLine, tsNormal, and wxEmptyString.
Referenced by Parser::ParseBufferForNamespaces().
bool ParserThread::ParseBufferForUsingNamespace | ( | const wxString & | buffer, |
wxArrayString & | result | ||
) |
Get the context "using namespace XXX" directive.
It is used to find the initial search scope before CC prompt a suggestion list. Need a critical section locker before call this function!
buffer | wxString to be parsed. |
result | the wxArrayString contains all the namespace names. |
Definition at line 392 of file parserthread.cpp.
References wxArrayString::Add(), ParserThreadOptions::bufferSkipBlocks, wxArrayString::Clear(), wxString::Clear(), ParserConsts::dcolon, Tokenizer::GetToken(), Tokenizer::InitFromBuffer(), IS_ALIVE, wxString::IsEmpty(), Tokenizer::IsOK(), ParserConsts::kw_namespace, ParserConsts::kw_using, m_EncounteredNamespaces, m_EncounteredTypeNamespaces, m_LastUnnamedTokenName, m_Options, m_ParsingTypedef, m_Str, m_Tokenizer, Tokenizer::NotEOF(), ParserConsts::opbrace, Tokenizer::PeekToken(), ParserConsts::semicolonclbrace, SkipBlock(), and SkipToOneOfChars().
Referenced by Parser::ParseBufferForUsingNamespace().
|
protected |
read <> as a whole token
Definition at line 3770 of file parserthread.cpp.
References _T, Tokenizer::GetToken(), wxString::Last(), m_Tokenizer, Tokenizer::NotEOF(), wxEmptyString, and wxT.
Referenced by DoParse().
|
protected |
handle class names, e.g., the code below
ancestor | class name = "AAA" this function reads the following "BBB" and "CCC", and regard them as derived classes of "AAA" |
Definition at line 3166 of file parserthread.cpp.
References _T, wxString::clear(), wxString::Clear(), ParserConsts::comma, CCLogger::DebugLog(), DoAddToken(), F(), CCLogger::Get(), Tokenizer::GetFilename(), Tokenizer::GetLineNumber(), Tokenizer::GetToken(), IS_ALIVE, wxString::IsEmpty(), Token::m_AncestorsString, m_LastParent, Token::m_Name, m_PointerOrRef, m_Str, m_Tokenizer, ParserConsts::ptr, RefineAnonymousTypeToken(), ParserConsts::semicolon, wxString::StartsWith(), tkClass, tkTypedef, TRACE, ParserConsts::underscore_chr(), Tokenizer::UngetToken(), wxString::wx_str(), and wxIsalpha().
Referenced by HandleClass(), and HandleTypedef().
|
protected |
eg: class ClassA{...} varA, varB, varC This function will read the "varA, varB, varC"
Definition at line 3111 of file parserthread.cpp.
References _T, ParserConsts::clarray, wxString::Clear(), ParserConsts::comma, CCLogger::DebugLog(), DoAddToken(), F(), CCLogger::Get(), Tokenizer::GetFilename(), Tokenizer::GetLineNumber(), Tokenizer::GetToken(), IS_ALIVE, wxString::IsEmpty(), m_LastParent, Token::m_Name, m_PointerOrRef, m_Str, m_Tokenizer, ParserConsts::oparray, ParserConsts::ptr, RefineAnonymousTypeToken(), ParserConsts::semicolon, SkipToOneOfChars(), wxString::StartsWith(), tkUndefined, tkVariable, TRACE, ParserConsts::underscore_chr(), wxString::wx_str(), and wxIsalpha().
Referenced by HandleClass().
|
private |
change an anonymous(unnamed) token's name to a human readable name, the m_Str is expect to store the unnamed token name, for example, for parsing the code
when we first find an anonymous token, which is named _UnnamedStruct1_2, after this function call, the anonymous token name will becomes struct1_abc, and m_Str will changed from _UnnamedStruct1_2 to struct1_abc.
Definition at line 3751 of file parserthread.cpp.
References _T, wxString::Contains(), m_FileIdx, Token::m_IsAnonymous, m_LastParent, m_Str, m_TokenTree, TokenTree::RenameToken(), and TokenExists().
Referenced by ReadClsNames(), and ReadVarNames().
|
private |
remove template arguments from an expression example: 'std::list<string>' will be separated into 'std::list' and '<string>'
expr | Complete expression with template arguments |
expNoArgs | Returned expression without template arguments |
templateArgs | The removed template arguments |
Definition at line 3688 of file parserthread.cpp.
References wxString::clear(), ParserConsts::gt, ParserConsts::gt_chr(), wxString::length(), ParserConsts::lt, ParserConsts::lt_chr(), and ParserConsts::space.
Referenced by HandleConditionalArguments(), and HandleForLoopArguments().
|
private |
this function just associate the formal template argument to actual argument For example, we have such code:
When handling the "bbb", we need to construct a TemplateMap, we store the map in the "bbb"'s member variable, which is "T"->"int".
Definition at line 3517 of file parserthread.cpp.
References _T, wxArrayString::GetCount(), Token::m_FullType, Token::m_TemplateArgument, m_TemplateArgument, Token::m_TemplateMap, Token::m_TemplateType, ResolveTemplateMap(), SplitTemplateActualParameters(), TRACE, and wxString::wx_str().
Referenced by DoParse(), HandleConditionalArguments(), HandleForLoopArguments(), HandleFunction(), and HandleTypedef().
|
private |
associate formal argument with actual template argument
Definition at line 3626 of file parserthread.cpp.
References _T, TokenTree::at(), wxString::Find(), TokenTree::FindMatches(), wxArrayString::GetCount(), Token::m_FullType, Token::m_TemplateType, Token::m_TokenKind, m_TokenTree, wxString::substr(), tkClass, tkTypedef, TRACE, wxString::Trim(), and wxNOT_FOUND.
Referenced by ResolveTemplateArgs().
|
protected |
skip the template argument
Definition at line 288 of file parserthread.cpp.
References Tokenizer::GetState(), Tokenizer::GetToken(), ParserConsts::gt, IS_ALIVE, wxString::IsEmpty(), ParserConsts::lt, m_Tokenizer, ParserConsts::semicolon, Tokenizer::SetState(), tsNormal, and Tokenizer::UngetToken().
Referenced by DoParse(), and HandleClass().
|
protected |
skip blocks {}
Definition at line 259 of file parserthread.cpp.
References Tokenizer::GetNestingLevel(), Tokenizer::GetState(), Tokenizer::GetToken(), IS_ALIVE, wxString::IsEmpty(), m_Tokenizer, Tokenizer::SetState(), and tsNormal.
Referenced by DoParse(), HandleFunction(), ParseBufferForNamespaces(), and ParseBufferForUsingNamespace().
|
protected |
Continuously eat the tokens until we meet one of the matching characters.
chars | wxString includes all the matching characters |
supportNesting | if true, we need to consider the "{" and "}" nesting levels when skipping, |
singleCharToken | if true, only single char tokens (like semicolon, brace etc.) are considered (speeds up parsing for queries like this) in this case, the function returned on a match by nesting/brace level preserved. |
Definition at line 232 of file parserthread.cpp.
References wxString::Find(), wxString::GetChar(), Tokenizer::GetNestingLevel(), Tokenizer::GetToken(), IS_ALIVE, wxString::IsEmpty(), wxString::length(), m_Tokenizer, ParserConsts::null(), and wxNOT_FOUND.
Referenced by CalcEnumExpression(), DoParse(), HandleClass(), HandleEnum(), HandleFunction(), HandleNamespace(), ParseBufferForNamespaces(), ParseBufferForUsingNamespace(), and ReadVarNames().
|
private |
Split actual template argument list.
Definition at line 3592 of file parserthread.cpp.
References _T, wxArrayString::Add(), ParserConsts::comma, wxArrayString::GetCount(), GetTemplateArgArray(), ParserConsts::gt, ParserConsts::lt, and TRACE.
Referenced by ResolveTemplateArgs().
|
private |
Split formal template argument list.
Definition at line 3573 of file parserthread.cpp.
References wxArrayString::Add(), wxArrayString::GetCount(), GetTemplateArgArray(), ParserConsts::kw_class, and ParserConsts::kw_typename.
Referenced by HandleClass().
|
private |
if parent is 0, then global namespace will be used, all tokens under parent scope are searched
name | the search key string |
parent | parent token pointer, we only search under the parent token scope |
kindMask | filter for the result token, only the specified type of tokens were matched |
Definition at line 1321 of file parserthread.cpp.
References TokenTree::at(), Token::m_Index, m_TokenTree, m_UsedNamespacesIds, TokenTree::TokenExists(), and wxNOT_FOUND.
Referenced by DoAddToken(), FindTokenFromQueue(), HandleClass(), HandleEnum(), HandleNamespace(), and RefineAnonymousTypeToken().
|
private |
Support function overloading.
Definition at line 1336 of file parserthread.cpp.
References TokenTree::at(), Token::m_Index, m_TokenTree, m_UsedNamespacesIds, TokenTree::TokenExists(), and wxNOT_FOUND.
|
private |
a wxString holding the parsing buffer, if it is a file in the hard disk, then this stands for the file name.
Otherwise, this is the in memory buffer content.
Definition at line 467 of file parserthread.h.
Referenced by InitTokenizer().
|
private |
for member funcs implementation or a function declaration below eg: int ClassA::FunctionB(); EncounteredNamespaces is 'ClassA'
Definition at line 444 of file parserthread.h.
Referenced by DoAddToken(), DoParse(), HandleFunction(), and ParseBufferForUsingNamespace().
|
private |
namespaces in function return types for a function declaration below: e.g.
: ClassC::returnValue ClassA::FunctionB(); m_EncounteredTypeNamespaces is 'ClassC'
Definition at line 451 of file parserthread.h.
Referenced by DoAddToken(), DoParse(), HandleFunction(), and ParseBufferForUsingNamespace().
|
private |
Definition at line 477 of file parserthread.h.
Referenced by HandleEnum().
|
private |
source file index on the "file map tree"
Definition at line 419 of file parserthread.h.
Referenced by DoAddToken(), FindTokenFromQueue(), HandleClass(), HandleEnum(), HandleNamespace(), InitTokenizer(), Parse(), and RefineAnonymousTypeToken().
|
private |
the file name of the parsing source
Definition at line 413 of file parserthread.h.
Referenced by HandleFunction(), HandleIncludes(), InitTokenizer(), and Parse().
|
private |
file size, actually the length of the wxString
Definition at line 416 of file parserthread.h.
Referenced by InitTokenizer().
|
private |
if true, means we are parsing a file which belongs to a C::B project, otherwise, we are parsing a file not belong to a C::B project(such as a system header file)
Definition at line 424 of file parserthread.h.
Referenced by DoAddToken(), FindTokenFromQueue(), and HandleIncludes().
|
private |
parent Token, for example, when you are parsing in the class member variables, m_LastParent holds a pointer to the current context, which is a token holding class name
Definition at line 405 of file parserthread.h.
Referenced by DoAddToken(), DoParse(), HandleClass(), HandleEnum(), HandleFunction(), HandleNamespace(), HandleTypedef(), ReadClsNames(), ReadVarNames(), and RefineAnonymousTypeToken().
|
private |
this member define the scope type of member variables, which is: public, private protected or undefined
Definition at line 410 of file parserthread.h.
Referenced by DoAddToken(), DoParse(), HandleClass(), and HandleNamespace().
|
private |
hold the previous token string
Definition at line 435 of file parserthread.h.
Referenced by DoParse().
|
private |
TODO: describe me here.
Definition at line 459 of file parserthread.h.
Referenced by DoParse(), HandleClass(), HandleEnum(), HandleTypedef(), and ParseBufferForUsingNamespace().
|
private |
parser options, see the ParserThreadOptions structure
Definition at line 438 of file parserthread.h.
Referenced by DoAddToken(), DoParse(), HandleClass(), HandleEnum(), HandleFunction(), HandleIncludes(), InitTokenizer(), ParserBase::Options(), Parse(), ParseBufferForUsingNamespace(), and ~ParserThread().
|
private |
a pointer to its parent Parser object, the Parserthread class has two place to communicate with Parser class.
m_Parent->ParseFile() when it see an include directive.
Definition at line 397 of file parserthread.h.
Referenced by HandleIncludes().
|
private |
this makes a difference in unnamed class/struct/enum handling
Definition at line 462 of file parserthread.h.
Referenced by HandleClass(), HandleEnum(), HandleTypedef(), Parse(), ParseBufferForNamespaces(), and ParseBufferForUsingNamespace().
|
private |
a pointer indicator or a references
Definition at line 470 of file parserthread.h.
Referenced by DoAddToken(), DoParse(), HandleClass(), HandleConditionalArguments(), HandleForLoopArguments(), HandleTypedef(), ReadClsNames(), and ReadVarNames().
|
private |
This is a very important member variables! It serves as a type stack, eg: parsing the statement: "unsigned int const varA;" we determine 'varA' is a token variable, until we searching to the last semicolon.
every token before 'varA' will be pushed to m_Str, at this time m_Str = "unsigned int const"
Definition at line 432 of file parserthread.h.
Referenced by DoAddToken(), DoParse(), GetTokenBaseType(), HandleClass(), HandleConditionalArguments(), HandleEnum(), HandleForLoopArguments(), HandleFunction(), HandleTypedef(), ParseBufferForUsingNamespace(), ReadClsNames(), ReadVarNames(), and RefineAnonymousTypeToken().
|
private |
Definition at line 475 of file parserthread.h.
Referenced by HandleClass().
|
private |
holds current template argument(s) when a template occurs
Definition at line 473 of file parserthread.h.
Referenced by DoAddToken(), DoParse(), GetTemplateArgs(), HandleClass(), HandleConditionalArguments(), HandleForLoopArguments(), HandleFunction(), HandleTypedef(), and ResolveTemplateArgs().
|
private |
if we regard the parserThread class as a syntax analyzer, then the Tokenizer class is regard as the lexer, which always feeds a wxString by calling m_Tokenizer.GetToken()
Definition at line 392 of file parserthread.h.
Referenced by CalcEnumExpression(), DoAddToken(), DoParse(), GetTemplateArgs(), HandleClass(), HandleConditionalArguments(), HandleEnum(), HandleForLoopArguments(), HandleFunction(), HandleIncludes(), HandleNamespace(), HandleTypedef(), InitTokenizer(), Parse(), ParseBufferForNamespaces(), ParseBufferForUsingNamespace(), ParserThread(), ReadAngleBrackets(), ReadClsNames(), ReadVarNames(), SkipAngleBraces(), SkipBlock(), and SkipToOneOfChars().
|
private |
a pointer to the token tree, all the tokens will be added to that tree structure
Definition at line 400 of file parserthread.h.
Referenced by CalcEnumExpression(), DoAddToken(), FindTokenFromQueue(), HandleIncludes(), InitTokenizer(), Parse(), ParserThread(), RefineAnonymousTypeToken(), ResolveTemplateMap(), and TokenExists().
|
private |
globally included namespaces by "using namespace" statement
Definition at line 456 of file parserthread.h.
Referenced by DoParse(), and TokenExists().