c++ - Boost Spirit X3 cannot compile repeat directive with variable factor

Question

I am trying to use the Boost Spirit X3 directive repeat with a repetition factor that is variable. The basic idea is that of a header + payload, where the header specifies the size of the payload. A simple example “3 1 2 3” is interpreted as header = 3, data= {1, 2, 3} (3 integers).

I could only find examples from the spirit qi documentation. It uses boost phoenix reference to wrap the variable factor: http://www.boost.org/doc/libs/1_50_0/libs/spirit/doc/html/spirit/qi/reference/directive/repeat.html

std::string str;
int n;
test_parser_attr("x0bHello World",
    char_[phx::ref(n) = _1] >> repeat(phx::ref(n))[char_], str);
std::cout << n << ',' << str << std::endl;  // will print "11,Hello World"

I wrote the following simple example for spirit x3 without luck:

#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <string>
#include <iostream>

namespace x3 = boost::spirit::x3;
using x3::uint_;
using x3::int_;
using x3::phrase_parse;
using x3::repeat;
using x3::space;
using std::string;
using std::cout;
using std::endl;

int main( int argc, char **argv )
{
  string data("3 1 2 3");
  string::iterator begin = data.begin();
  string::iterator end = data.end();

  unsigned int n = 0;

  auto f = [&n]( auto &ctx ) { n = x3::_attr(ctx); };
  bool r = phrase_parse( begin, end, uint_[f] >> repeat(boost::phoenix::ref(n))[int_], space );
  if ( r && begin == end  )
    cout << "Parse success!" << endl; 
  else
    cout << "Parse failed, remaining: " << string(begin,end) << endl;

  return 0;
}

Compiling the code above with boost 1.59.0 and clang++ (flags: -std=c++14) gives the following:

boost_1_59_0/boost/spirit/home/x3/directive/repeat.hpp:72:47: error: no matching constructor for

      initialization of 'proto_child0' (aka 'boost::reference_wrapper<unsigned int>')

            typename RepeatCountLimit::type i{};

If I hardcode repeat(3) instead of repeat(boost::phoenix::ref(n)) it works properly, but it is not a possible solution since it should support a variable repetition factor.

Compilation with repeat(n) completes successfully, but it fails parsing with the following output: “Parse failed, remaining: 1 2 3"

Looking at the source code for boost/spirit/home/x3/directive/repeat.hpp:72 it calls the empty constructor for template type RepeatCountLimit::type variable i and then assign during the for loop, iterating over min and max. However since the type is a reference it should be initialized in the constructor, so compilation fails. Looking at the equivalent source code from the previous library version boost/spirit/home/qi/directive/repeat.hpp:162 it is assigned directly:

        typename LoopIter::type i = iter.start();

I am not sure what I am doing wrong here, or if x3 currently does not support variable repetition factors. I would appreciate some help solving this issue. Thank you.

Answer 1

From what I gather, reading the source and the mailing list, Phoenix is not integrated into X3 at all: the reason being that c++14 makes most of it obsolete.

I agree that this leaves a few spots where Qi used to have elegant solutions, e.g. eps(DEFERRED_CONDITION), lazy(*RULE_PTR) (the Nabialek trick), and indeed, this case.

Spirit X3 is still in development, so we might see this added1

For now, Spirit X3 has one generalized facility for stateful context. This essentially replaces locals<>, in some cases inherited arguments, and can be /made to/ validate the number of elements in this particular case as well:

• x3::with2

Here's how you could use it:

with<_n>(std::ref(n)) 
    [ omit[uint_[number] ] >> 
    *(eps [more] >> int_) >> eps [done] ]

Here, _n is a tag type that identifies the context element for retrieval with get<_n>(cxtx).

Note, currently we have to use a reference-wrapper to an lvalue n because with<_n>(0u) would result in constant element inside the context. I suppose this, too, is a QoI that may be lifted as X# matures

Now, for the semantic actions:

unsigned n;
struct _n{};

auto number = [](auto &ctx) { get<_n>(ctx).get() = _attr(ctx); };

This stores the parsed unsigned number into the context. (In fact, due to the ref(n) binding it's not actually part of the context for now, as mentioned)

auto more   = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) >  _val(ctx).size(); };

Here we check that we're actually not "full" - i.e. more integers are allowed

auto done   = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) == _val(ctx).size(); };

Here we check that we're "full" - i.e. no more integers are allowed.

Putting it all together:

Live On Coliru

#include <string>
#include <iostream>
#include <iomanip>

#include <boost/spirit/home/x3.hpp>

int main() {
    for (std::string const input : { 
            "3 1 2 3", // correct
            "4 1 2 3", // too few
            "2 1 2 3", // too many
            // 
            "   3 1 2 3   ",
        })
    {
        std::cout << "
Parsing " << std::left << std::setw(20) << ("'" + input + "':");

        std::vector<int> v;

        bool ok;
        {
            using namespace boost::spirit::x3;

            unsigned n;
            struct _n{};

            auto number = [](auto &ctx) { get<_n>(ctx).get() = _attr(ctx); };
            auto more   = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) >  _val(ctx).size(); };
            auto done   = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) == _val(ctx).size(); };

            auto r = rule<struct _r, std::vector<int> > {} 
                  %= with<_n>(std::ref(n)) 
                        [ omit[uint_[number] ] >> *(eps [more] >> int_) >> eps [done] ];

            ok = phrase_parse(input.begin(), input.end(), r >> eoi, space, v);
        }

        if (ok) {
            std::copy(v.begin(), v.end(), std::ostream_iterator<int>(std::cout << v.size() << " elements: ", " "));
        } else {
            std::cout << "Parse failed";
        }
    }
}

Which prints:

Parsing '3 1 2 3':          3 elements: 1 2 3 
Parsing '4 1 2 3':          Parse failed
Parsing '2 1 2 3':          Parse failed
Parsing '   3 1 2 3   ':    3 elements: 1 2 3 

---

1 lend your support/voice at the [spirit-general] mailing list :)

2 can't find a suitable documentation link, but it's used in some of the samples