• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    公众号

Java Token类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Java中com.sonar.sslr.api.Token的典型用法代码示例。如果您正苦于以下问题:Java Token类的具体用法?Java Token怎么用?Java Token使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



Token类属于com.sonar.sslr.api包,在下文中一共展示了Token类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Java代码示例。

示例1: visitToken

import com.sonar.sslr.api.Token; //导入依赖的package包/类
@Override
public void visitToken(Token token) {
  if (token.isGeneratedCode()) {
    return;
  }

  if (previousToken != null && previousToken.getLine() != token.getLine()) {
    // Note that AbstractLineLengthCheck doesn't support tokens which span multiple lines - see SONARPLUGINS-2025
    String[] lines = previousToken.getValue().split("\r?\n|\r", -1);
    int length = previousToken.getColumn();
    for (String line : lines) {
      length += line.length();
      if (length > getMaximumLineLength()) {
        // Note that method from AbstractLineLengthCheck generates other message - see SONARPLUGINS-1809
        getContext().createLineViolation(this,
          "The line contains {0,number,integer} characters which is greater than {1,number,integer} authorized.",
          previousToken.getLine(),
          length,
          getMaximumLineLength());
      }
      length = 0;
    }
  }
  previousToken = token;
}
 
开发者ID:antowski,项目名称:sonar-onec,代码行数:26,代码来源:LineLengthCheck.java


示例2: visitToken

import com.sonar.sslr.api.Token; //导入依赖的package包/类
@Override
public void visitToken(Token token) {
  if (token.getType().equals(PuppetTokenType.SINGLE_QUOTED_STRING_LITERAL) || token.getType().equals(PuppetTokenType.DOUBLE_QUOTED_STRING_LITERAL)) {
    // case: string literal, including doc string
    highlight(token, TypeOfText.STRING);

  } else if (token.getType() instanceof PuppetKeyword) {
    // case: keyword
    highlight(token, TypeOfText.KEYWORD);
  }

  for (Trivia trivia : token.getTrivia()) {
    // case: comment
    highlight(trivia.getToken(), TypeOfText.COMMENT);
  }
}
 
开发者ID:iwarapter,项目名称:sonar-puppet,代码行数:17,代码来源:PuppetHighlighter.java


示例3: TokenLocation

import com.sonar.sslr.api.Token; //导入依赖的package包/类
public TokenLocation(Token token) {
  this.startLine = token.getLine();
  this.startLineOffset = token.getColumn();

  String value = token.getValue();
  String[] lines = value.split("\r\n|\n|\r", -1);

  if (lines.length > 1) {
    endLine = token.getLine() + lines.length - 1;
    endLineOffset = lines[lines.length - 1].length();

  } else {
    this.endLine = this.startLine;
    this.endLineOffset = this.startLineOffset + token.getValue().length();
  }
}
 
开发者ID:iwarapter,项目名称:sonar-puppet,代码行数:17,代码来源:TokenLocation.java


示例4: consume

import com.sonar.sslr.api.Token; //导入依赖的package包/类
@Override
public boolean consume(CodeReader code, Lexer lexer) {
  if (code.popTo(matcher, tmpBuilder) > 0) {
    String word = tmpBuilder.toString();
    String wordOriginal = word;
    if (!caseSensitive) {
      word = word.toUpperCase();
    }

    TokenType keywordType = keywordsMap.get(word);
    Token token = tokenBuilder
      .setType(keywordType == null ? NAME : keywordType)
      .setValueAndOriginalValue(word, wordOriginal)
      .setURI(lexer.getURI())
      .setLine(code.getPreviousCursor().getLine())
      .setColumn(code.getPreviousCursor().getColumn())
      .build();

    lexer.addToken(token);

    tmpBuilder.delete(0, tmpBuilder.length());
    return true;
  }
  return false;
}
 
开发者ID:iwarapter,项目名称:sonar-puppet,代码行数:26,代码来源:NameAndKeywordChannel.java


示例5: visitToken

import com.sonar.sslr.api.Token; //导入依赖的package包/类
public void visitToken(Token token) {
  if (!token.isGeneratedCode()) {
    if (previousToken != null && previousToken.getLine() != token.getLine()) {
      int length = previousToken.getColumn() + previousToken.getValue().length();
      if (length > getMaximumLineLength()) {
        // Note that method from AbstractLineLengthCheck generates other message - see SONARPLUGINS-1809
        getContext().createLineViolation(this,
            "The line contains {0,number,integer} characters which is greater than {1,number,integer} authorized.",
            previousToken.getLine(),
            length,
            getMaximumLineLength());
      }
    }
    previousToken = token;
  }
}
 
开发者ID:Ne0s,项目名称:sonar-plsql,代码行数:17,代码来源:LineLengthCheck.java


示例6: visitToken

import com.sonar.sslr.api.Token; //导入依赖的package包/类
@Override
public void visitToken(Token token) {

    if (token.getType().equals(GenericTokenType.EOF)) {
        return;
    }
    
    if(token.getType() == GenericTokenType.UNKNOWN_CHAR){
        linesOfCode.add(token.getLine());
    }

    if(token.getType() == GenericTokenType.COMMENT){
        linesOfComments.add(token.getLine());
    }
}
 
开发者ID:antowski,项目名称:sonar-onec,代码行数:16,代码来源:FileLinesVisitor.java


示例7: tokenize

import com.sonar.sslr.api.Token; //导入依赖的package包/类
/**
 * Builds and stores a {@link TokenEntry} from the source code.
 *
 * @param source source code.
 * @param cpdTokens tokens.
 * @throws IOException when it can't read a source file.
 */
@Override
public void tokenize(SourceCode source, Tokens cpdTokens) throws IOException {
    Lexer lexer = ApexLexer.create(config);
    String fileName = source.getFileName();
    List<Token> tokens = lexer.lex(new File(fileName));
    tokens.forEach(token -> {
        TokenEntry cpdToken = new TokenEntry(getTokenImage(token), fileName, token.getLine());
        cpdTokens.add(cpdToken);
    });
    cpdTokens.add(TokenEntry.getEOF());
}
 
开发者ID:fundacionjala,项目名称:enforce-sonarqube-plugin,代码行数:19,代码来源:ApexTokenizer.java


示例8: startElement

import com.sonar.sslr.api.Token; //导入依赖的package包/类
public void startElement(String uri, String name, String qName, Attributes atts) {
	int line = locator.getLineNumber();
	int column = locator.getColumnNumber();
	if(FlowLexer.FlowTypes.isInEnum("START_" + name.toUpperCase())){
		logger.debug("Start element: " + qName + "[" + line + "," + column + "]" + "[START_" + name.toUpperCase() + "]");
		Token token = tokenBuilder.setType(FlowLexer.FlowTypes.valueOf("START_" + name.toUpperCase())).setValueAndOriginalValue(name.toUpperCase(),name)
				.setURI(lex.getURI()).setLine(line).setColumn(0).build();
		lex.addToken(token);
		// CHECK THE ATTRIBUTES
		for(int i=0; i<atts.getLength();i++){
			if(FlowLexer.FlowAttTypes.isInEnum(atts.getQName(i).toUpperCase())){
				token = tokenBuilder.setType(FlowLexer.FlowAttTypes.getEnum(atts.getQName(i).toUpperCase())).setValueAndOriginalValue(atts.getValue(i).toUpperCase(),atts.getValue(i))
						.setURI(lex.getURI()).setLine(line).setColumn(0).build();
				lex.addToken(token);
				logger.debug("TOKEN " + token.getValue() + "[" + token.getLine() + "," + token.getColumn() + "]");
			}else{
				token = tokenBuilder.setType(GenericTokenType.IDENTIFIER).setValueAndOriginalValue(atts.getQName(i))
						.setURI(lex.getURI()).setLine(line).setColumn(0).build();
				lex.addToken(token);
				logger.debug("IDENTIFIER " + token.getValue() + "[" + token.getLine() + "," + token.getColumn() + "]");
				token = tokenBuilder.setType(GenericTokenType.LITERAL).setValueAndOriginalValue(atts.getValue(i))
						.setURI(lex.getURI()).setLine(line).setColumn(0).build();
				lex.addToken(token);
				logger.debug("LITERAL " + token.getValue() + "[" + token.getLine() + "," + token.getColumn() + "]");
			}
		}
	}
}
 
开发者ID:I8C,项目名称:sonar-flow-plugin,代码行数:29,代码来源:FlowContentHandler.java


示例9: visitToken

import com.sonar.sslr.api.Token; //导入依赖的package包/类
@Override
public void visitToken(Token token) {
  Iterator iterator = token.getTrivia().iterator();
  while (iterator.hasNext()) {
    Trivia trivia = (Trivia) iterator.next();
    if (trivia.isComment() && pattern.matcher(trivia.getToken().getOriginalValue()).matches()) {
      addIssue(trivia.getToken().getLine(), this, "Use starting comment token '#' instead.");
    }
  }
}
 
开发者ID:iwarapter,项目名称:sonar-puppet,代码行数:11,代码来源:CommentConventionCheck.java


示例10: visitToken

import com.sonar.sslr.api.Token; //导入依赖的package包/类
public void visitToken(Token token) {
  for (Trivia trivia : token.getTrivia()) {
    String comment = trivia.getToken().getOriginalValue();
    if (StringUtils.containsIgnoreCase(comment, pattern)) {
      String[] lines = comment.split("\r\n?|\n");

      for (int i = 0; i < lines.length; i++) {
        if (StringUtils.containsIgnoreCase(lines[i], pattern) && !isLetterAround(lines[i], pattern)) {
          check.addIssue(trivia.getToken().getLine() + i, check, message);
        }
      }
    }
  }
}
 
开发者ID:iwarapter,项目名称:sonar-puppet,代码行数:15,代码来源:CommentContainsPatternChecker.java


示例11: tokenize

import com.sonar.sslr.api.Token; //导入依赖的package包/类
@Override
public final void tokenize(SourceCode source, Tokens cpdTokens) {
  Lexer lexer = PuppetLexer.create(new PuppetConfiguration(charset));
  String fileName = source.getFileName();
  List<Token> tokens = lexer.lex(new File(fileName));
  for (Token token : tokens) {
    if (!token.getType().equals(PuppetTokenType.NEWLINE) && !token.getType().equals(PuppetTokenType.DEDENT) && !token.getType().equals(PuppetTokenType.INDENT)) {
      TokenEntry cpdToken = new TokenEntry(getTokenImage(token), fileName, token.getLine());
      cpdTokens.add(cpdToken);
    }
  }
  cpdTokens.add(TokenEntry.getEOF());
}
 
开发者ID:iwarapter,项目名称:sonar-puppet,代码行数:14,代码来源:PuppetTokenizer.java


示例12: visitToken

import com.sonar.sslr.api.Token; //导入依赖的package包/类
/**
 * {@inheritDoc}
 */
@Override
public void visitToken(Token token) {
  if (!token.getType().equals(EOF)) {
    String[] tokenLines = token.getValue().split("\n", -1);

    int firstLineAlreadyCounted = lastTokenLine == token.getLine() ? 1 : 0;
    getContext().peekSourceCode().add(metric, (double) tokenLines.length - firstLineAlreadyCounted);

    lastTokenLine = token.getLine() + tokenLines.length - 1;
  }

}
 
开发者ID:iwarapter,项目名称:sonar-puppet,代码行数:16,代码来源:PuppetLinesOfCodeVisitor.java


示例13: consume

import com.sonar.sslr.api.Token; //导入依赖的package包/类
@Override
public boolean consume(CodeReader code, Lexer output) {
  if (code.peek() == '/') {
    Token lastToken = getLastToken(output);
    if (lastToken == null || lastToken.getType().equals(PuppetKeyword.NODE) || guessNextIsRegexp(lastToken.getValue())) {
      return delegate.consume(code, output);
    }
  }
  return false;
}
 
开发者ID:iwarapter,项目名称:sonar-puppet,代码行数:11,代码来源:PuppetRegexpChannel.java


示例14: matchSuccess

import com.sonar.sslr.api.Token; //导入依赖的package包/类
@Override
public void matchSuccess(final MatchSuccessEvent<Token.Builder> event)
{
    final MatcherContext<Token.Builder> context = event.getContext();
    if (!context.inPredicate())
        consumed = Math.max(consumed, context.getCurrentIndex());
    if (context.getLevel() != 0)
        return;
    if (context.getMatcher() != rootMatcher)
        throw new IllegalStateException("was expecting root rule here");
}
 
开发者ID:litesolutions,项目名称:sonar-sslr-grappa,代码行数:12,代码来源:CodeReaderListener.java


示例15: matchFailure

import com.sonar.sslr.api.Token; //导入依赖的package包/类
@Override
public void matchFailure(final MatchFailureEvent<Token.Builder> event)
{
    final MatcherContext<Token.Builder> context = event.getContext();
    if (context.getLevel() != 0)
        return;
    if (context.getMatcher() != rootMatcher)
        throw new IllegalStateException("was expecting root rule here");
}
 
开发者ID:litesolutions,项目名称:sonar-sslr-grappa,代码行数:10,代码来源:CodeReaderListener.java


示例16: afterParse

import com.sonar.sslr.api.Token; //导入依赖的package包/类
@Override
public void afterParse(final PostParseEvent<Token.Builder> event)
{
    final int length = reader.length();

    /*
     * We want a match
     */

    final ParsingResult<Token.Builder> result = event.getResult();
    if (!result.isSuccess())
        throw new GrappaException("match failure (consumed: "
            + consumed + " out of " + length + ')');

    /*
     * Check that we did consume all the text
     */

    if (consumed != length)
        throw new GrappaException("was expecting to fully match, but only "
            + consumed + " chars were matched out of " + length);

    final ValueStack<Token.Builder> stack = result.getValueStack();

    final URI uri = lexer.getURI();
    final int size = stack.size();
    Token token;

    for (int index = size - 1; index >= 0; index--) {
        token = stack.peek(index).setURI(uri).build();
        if (token.getType() == GenericTokenType.COMMENT)
            lexer.addTrivia(Trivia.createComment(token));
        else
            lexer.addToken(token);
    }
}
 
开发者ID:litesolutions,项目名称:sonar-sslr-grappa,代码行数:37,代码来源:CodeReaderListener.java


示例17: consume

import com.sonar.sslr.api.Token; //导入依赖的package包/类
@Override
public boolean consume(final CodeReader code, final Lexer output)
{
    final InputBuffer buffer = new CodeReaderInputBuffer(code);

    final ListeningParseRunner<Token.Builder> runner
        = new ListeningParseRunner<>(rule);

    suppliers.stream().map(supplier -> supplier.create(code, output))
        .forEach(runner::registerListener);

    runner.run(buffer);

    /*
     * Because of the CodeReaderListener here, we know that we have consumed
     * the full input text; if this isn't the case (because of a parsing
     * failure or because not all the input was consumed), an exception will
     * have been thrown.
     *
     * We therefore pop() all the contents of the reader at this point...
     */

    final int length = code.length();

    for (int i = 0; i < length; i++)
        code.pop();

    return true;
}
 
开发者ID:litesolutions,项目名称:sonar-sslr-grappa,代码行数:30,代码来源:GrappaChannel.java


示例18: pushToken

import com.sonar.sslr.api.Token; //导入依赖的package包/类
public boolean pushToken(final TokenType tokenType)
{
    final Context<Token.Builder> context = getContext();
    final int startIndex = context.getMatchStartIndex();
    final Position position
        = context.getInputBuffer().getPosition(startIndex);

    final Token.Builder token = Token.builder()
        .setValueAndOriginalValue(match())
        .setLine(position.getLine())
        .setColumn(position.getColumn())
        .setType(tokenType);

    return push(token);
}
 
开发者ID:litesolutions,项目名称:sonar-sslr-grappa,代码行数:16,代码来源:SonarParserBase.java


示例19: tokenize

import com.sonar.sslr.api.Token; //导入依赖的package包/类
public final void tokenize(SourceCode source, Tokens cpdTokens) {
	Lexer lexer = PlSQLLexer.create(new PlSQLConfiguration(charset));
	String fileName = source.getFileName();
	List<Token> tokens = lexer.lex(new File(fileName));
	for (Token token : tokens) {
		TokenEntry cpdToken = new TokenEntry(getTokenImage(token),
				fileName, token.getLine());
		cpdTokens.add(cpdToken);
	}
	cpdTokens.add(TokenEntry.getEOF());
}
 
开发者ID:Ne0s,项目名称:sonar-plsql,代码行数:12,代码来源:PlSQLTokenizer.java


示例20: visitToken

import com.sonar.sslr.api.Token; //导入依赖的package包/类
public void visitToken(Token token) {
  if (token.getType().equals(GenericTokenType.EOF)) {
    return;
  }

  linesOfCode.add(token.getLine());
  List<Trivia> trivias = token.getTrivia();
  for (Trivia trivia : trivias) {
    if (trivia.isComment()) {
      linesOfComments.add(trivia.getToken().getLine());
    }
  }
}
 
开发者ID:Ne0s,项目名称:sonar-plsql,代码行数:14,代码来源:FileLinesVisitor.java



注:本文中的com.sonar.sslr.api.Token类示例整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Java PickerFragment类代码示例发布时间:2022-05-23
下一篇:
Java SwiftUnsupportedFeatureException类代码示例发布时间:2022-05-23
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap