many and count combinators when the output type is zero sizedmany_m_n to 64kiBParser implementors instead of only functionsTuple parsing for the unit type as a special caseErrorConvert on the unit type to make it usable as error type for bits parsersThis release fixes dependency compilation issues and strengthen the minimum supported Rust version (MSRV) policy. This is also the first release without the macros that were used since nom's beginning.
take_until1 combinatorto_owned implementationsfail: a parser that always fail, useful as default condition in other combinatorscharacter::streaming and character::complete modules, there are parsers named i8, u16, u32, u64, u128 and u8 ,u16, u32, u64, u128 that recognize decimal digits and directly convert to a number in the target size (checking for max int size)BitSlice input type from bitvec has been moved into the nom-bitvec crate. nom does not depend on bitvec nowErrorKind::PArseTo was not needed anymoresplit_at_position* functions should now be guaranteed panic freelexical-core crate used for float parsing has now been replaced with minimal-lexical: the new crate is faster to compile, faster to parse, and has no dependenciesescaped combinatormany_m_n now fails if min > maxThis release was done thanks to the hard work of (by order of appearance in the commit list):
This release was done thanks to the hard work of (by order of appearance in the commit list):
This release was done thanks to the hard work of (by order of appearance in the commit list):
This release was done thanks to the hard work of (by order of appearance in the commit list):
std::error::Error on VerboseErrorThis release was done thanks to the hard work of (by order of appearance in the commit list):
ErrorConvert implementation for VerboseErrorfold_many* now accept FnMut for the accumulation functionlength_count#[deprecated] attribute was removed from traits because it does not compile anymore on nightlyFnMutThis release was done thanks to the hard work of (by order of appearance in the commit list):
This release is a more polished version of nom 5, that came with a focus on function parsers, by relaxing the requirements: combinators will return a impl FnMut instead of impl Fn, allowing closures that change their context, and parsers can be any type now, as long as they implement the new Parser trait. That parser trait also comes with a few helper methods.
Error management was often a pain point, so a lot of work went into making it easier. Now it integrates with std:error::Error, the IResult::finish() method allows you to convert to a more usable type, the into combinator can convert the error type if there's a From implementation, and there are more specific error traits like ContextError for the context combinator, and FromExternalError for map_res. While the VerboseError type and its convert_error function saw some changes, not many features ill be added to it, instead you are encouraged to build the error type that corresponds to your needs if you are building a language parser.
This version also integrates with the excellent bitvec crate for better bit level parsing. This part of nom was not great and a bit of a hack, so this will give better options for those parsers.
At last, documentation! There are now more code examples, functions and macros that require specific cargo features are now clearly indicated, and there's a new recipes module containing example patterns.
alloc or std features)regexp_macros cargo featurecontext combinator is not linked to ParseError anymore, instead it come with its own ContextError traitNeeded::Size now contains a NonZeroUsize, so we can reduce the structure's size by 8 bytes. When upgrading, Needed::Size(number) can be replaced with Needed::new(number)Parser trait, so parsers can be something else than a function. This trait also comes with combinator methods like map, flat_map, or. Since it is implemented on Fn* traits, it should not affect existing code too muchimpl Fn now return a impl FnMut to allow parser closures that capture some mutable value from the contextseparated_list is now separated_list0methods modulewhitespace modulenom::error::Error) instead of a tupleFromExternalError allows wrapping the error returned by the function in the map_res combinatordbg! macro to avoid conflicts with std::dbg!separated_list now allows empty elementsfill: attempts to fill the output slice passed as argumentsuccess: returns a value without consuming the inputsatisfy: checks a predicate over the next charactereof function combinatorconsumed: returns the produced value and the consumed inputlength_count function combinatorinto: converts a parser's output and error values if From implementations are availableIResult::finish(): converts a parser's result to Result<(I, O), E> by removing the distinction between Error and Failure and panicking on Incompleteu16, i32, etc, with configurable endiannessis_newline functionstd::error::Error implementation for nom's error typesBitSlice type from the bitvec cratealloc featureconvert_error accepts a type that derefs to &strfold_many0c in the fold_many0 macromap implementation for errorsErr now has a map functionerror::context() available without alloc featureconvert_errorcompile_error macro usagestd::error::Error, std::fmt::Display, Eq, ToOwned implementations for errorsToUsizeconvert_error optimizationalt optimizationmany0_m_n fixesmany0_m_n now supports the n=1 casecutpeek! macro reimplementationvalue!This version comes with a complete rewrite of nom internals to use functions as a base for parsers, instead of macros. Macros have been updated to use functions under the hood, so that most existing parsers will work directly or require minimal changes.
The CompleteByteSlice and CompleteStr input types were removed. To get different behaviour related to streaming or complete input, there are different versions of some parsers in different submodules, like nom::character::streaming::alpha0 and nom::character::complete::alpha0.
The verbose-errors feature is gone, now the error type is decided through a generic bound. To get equivalent behaviour to verbose-errors, check out nom::error::VerboseError
escaped and escaped_transform fixesVerboseError type accumulates position info and error codes, and can generate a trace with span informationlexical-core crate is now used by default (through the lexical compilation feature) to parse floats from textcond will now return the error of the parser instead of Nonealpha*, digit*, hex_digit*, alphanumeric* now recognize only ASCII characters_s suffix), the normal version can be used insteadverbose-errors is not needed anymore, now the error type can be decided when writing the parsers, and parsers provided by nom are generic over the error typeAtEof, CompleteByteSlice and CompleteStr are gone, instead some parsers are specialized to work on streaming or complete input, and provided in different modules*1 version: eol, alpha, digit, hex_digit, oct_digit, alphanumeric, space, multispacecount_fixed macrowhitespace::sp can be replaced by character::complete::multispace0take_until_either, take_until_either1, take_until_either_and_consume and take_until_either_and_consume1: they can be replaced with is_not (possibly combined with something else)take_until_and_consume, take_until_and_consume1: they can be replaced with take_until combined with takesized_buffer and length_bytes!: they can be replaced with the length_data functionnon_empty, begin and rest_s functioncond_reduce!, cond_with_error!, closure!, apply, map_res_err!, expr_opt!, expr_res!alt_complete, separated_list_complete, separated_nonempty_list_completebuild.rs file to the packagedo_parse due to compile_error macro usagemany0_count and many1_count to count applications of a parser instead of accumulating its results in a Vecf64 parsing does not use transmute anymoreadd_return_errorrest_len parseru128 and i128 parserspub(crate) supportmap_res_err!rest_len parser, returns the length of the remaining inputparse_to has its own error code nowu128 and i128 parsers in big and little endian modespub(crate) syntaxmap_res_err! combinator that appends the error of its argument function in verbose errors modebytes combinator was not compiling in some casesadd_return_error, the provided error code is now evaluated only oncefold_many1 will now transmit a Failure instead of transforming it to an Errorfloat and double now work on all of nom's input types (&[u8], &str, CompleteByteSlice, CompleteStr and any type that implements the required traits). float_s and double_s got the same modification, but are now deprecatedCompleteByteSlice and CompleteStr get a small optimization by inlining some functionsAtEof traitescaped* combinatorsInputIter trait for &[u8]input_len() usage fixesswitch!Deref on CompleteStr and CompleteByteSliceparse_to! fixesCompleteStr and CompleteByteSliceCompleteStrAsRef<str> on CompleteStrstd::Error on nom:ErrSpecial thanks to @corkami for the logo :)
IResult type now becomes a Result from the standard libraryIncomplete now returns the additional data size needed, not the total data size needederror_position and other such macros were swapped to be more consistent with the rest of nomstd::convert::From<u32>not! combinator returns unit ()take_* combinators are now more coherent and stricter, see commit 484f6724ea3ccb for more informationmany0 and other related parsers will now return Incomplete if the reach the end of input without an error of the child parser. They will also return Incomplete on an empty inputsep! combinator for whitespace only consumes whitespace in the prefix, while the ws! combinator takes care of consuming the remaining whitespaceAtEof trait for input type: indicate if we can get more input data later (related to streaming parsers and Incomplete handling)escaped* parsers now support the &strinput typeFailure error variant represents an unrecoverable error, for which alt and other combinators will not try other branches. This error means we got in the right part of the code (like, a prefix was checked correctly), but there was an error in the following partsCompleteByteSlice and CompleteStr input types consider there will be no more refill of the input. They fixed the Incomplete related issues when we have all of the dataexact!() combinator will fail if we did not consume the whole inputtake_while_m_n! combinator will match a specified number of charactersErrorKind::TakeUntilAndConsume1recognize_float parser will match a float number's characters, but will not transform to a f32 or f64alpha and other basic parsers are now much stricter about partial inputs. We also introduce the *0 and *1 versions of those parsersnamed_args can now specify the input type as wellHexDisplay is now implemented for &stralloc featureInputTakeAtposition trait allows specialized implementations of parsers like take_while!error_code and error_node macros are not used anymoreanychar! now works correctly with multibyte characterstake_until_and_consume1! no longer results in “no method named `find_substring`” and “no method named `slice`” compilation errorstake_until_and_consume1! returns the correct Incomplete(Needed) amountno_std compiles properly, and nom can work with alloc tooparse_to! now consumes its inputalt and other combinators will now clone the input if necessary. If the input is already Copy there is no performance impactrest parser now works on various input typesInputIter::Item for &[u8] is now a u8 directly, not a referencecompile_error macro to return a compile time error if there was a syntax issuealt_complete fixesalt_complete could return Incompleteinto_error_kind method can be used to transform any error to a common value. This helps when the library is included multiple times as dependency with different feature setsbytes combinatorseparated_listanychar can now work on any input typelength_bytes is now an alias for length_dataone_of, none_of and char will now index correctly UTF-8 characterscompiler_error macro is now correctly exportedbytes combinator transforms a bit stream back to a byte slice for child parserscount no longer preallocates its vectoralt should now work with whitespace parsingmap should not make type inference errors anymoreseparated_list_complete and separated_nonempty_list_complete will treat incomplete from sub parsers as errorIncomplete related refactorsbe_u24IResultCopy types as inputInputIternamed_argsaltalt and alt_completeescaped, separated_list and separated_nonempty_list can now return Incomplete when necessaryInputIter does not require AsChar on its Item type anymorecore feature that was putting nom in no_std mode has been removed. There is now a std feature, activated by default. If it is not activated, nom is in no_stdverbose-errors mode, the error list is now stored in a Vec instead of a box based linked listchain! has finally been removedEndianness now implements Debug, PartialEq, Eq, Clone and CopyCopystr is now available in no_std modeFileProducer will be marked as Eof on full buffernamed_args! now has lifetimes that cannot conflict with the lifetimes from other argumentsbe_u24: big endian 24 bit unsigned integer parsingIResult now has a unwrap_or methodIResult::Incomplete, the addition could overflow (it is stored as a usize). This would apparently not result in any security vulnerability on release codenamed_argsIResult, adding the option of default cases in switch!, adding support for cargo-travishex_u32chain! as deprecatedIResult gets an or() methodtake_until1, take_until_and_consume1, take_till1! and take_till1_s! require at least 1 characterhex_u32 accepts uppercase digits as welltake_while1 returns Incomplete on empty inputswitch! can now take a default casenamed_args! now imports IResult directlySlicenamed_args!verify! uses a first parser, then applies a function to check that its result satisfies some conditionsnamed_args! creates a parser function that can accept other arguments along with the inputparse_to! will use the parse method from FromStr to parse a value. It will automatically translate the input to a string if necessaryfloat, float_s, double, double_s can recognize floating point numbers in textescaped! will now return Incomplete if neededpermutation! supports up to 20 child parsersBugfix release
Warning: there is a small breaking change, add_error! is renamed to add_return_error!. This was planned for the 2.0 release but was forgotten. This is a small change in a feature that not many people use, for a release that is not yet widely in use, so there will be no 3.0 release for that change.
add_error! mixuptag_no_case! was not working at all for byte slicesadd_error! has been renamed to add_return_error!not! combinator now accepts functionstag_no_case! is now working as accepted (before, it accepted everything)The 2.0 release is one of the biggest yet. It was a good opportunity to clean up some badly named combinators and fix invalid behaviours.
Since this version introduces a few breaking changes, an upgrade documentation is available, detailing the steps to fix the most common migration issues. After testing on a set of 30 crates, most of them will build directly, a large part will just need to activate the “verbose-errors” compilation feature. The remaining fixes are documented.
This version also adds a lot of interesting features, like the permutation combinator or whitespace separated formats support.
IResult => Result conversion work, making AsChar's method more consistent, and adding many_till!Offset on &strlength_value! and length_bytes!named_attr!crates.io badgetake_s!Error and Display on ErrorKind and detecting incorrect UTF-8 string indexingsrc/traits.rs: InputLength, InputIter, InputTake, Compare, FindToken, FindSubstring, Slicedoc/ directory. They are markdown files that you can build with cargo-external-docws! combinator, you can automatically introduce whitespace parsers between all parsers and combinatorspermutation! combinator applies its child parsers in any order, as long as they all succeed once, and return a tuple of the resultsdo_parse! is a simpler alternative to chain!, which is now deprecatedIResult in a std::result::Resultlength_data! parses a length, and returns a subslice of that lengthtag_no_case! provides case independent comparison. It works nicely, without any allocation, for ASCII strings, but for UTF-8 strings, it defaults to an unsatisfying (and incorrect) comparison by lowercasing both stringsnamed_attr! creates functions like named! but can add attributes like documentationmany_till! applies repeatedly its first child parser until the second succeedseof function was replaced with the eof! macroerror! and add_error! were replaced with return_error! and add_return_error! to fix the name conflict with the log crateoffset() method is now in the Offset traitlength_value! has been renamed to length_count!. The new length_value! selects a slice and applies the second parser once on that sliceAsChar::is_0_to_9 is now AsChar::is_dec_digitcount!, count_fixed! and length_*! combinator calculate incomplete data needs correctlyeol, line_ending and not_line_ending now have a consistent behaviour that works correctly with incomplete datatake_s! didn't correctly handle the case when the slice is exactly the right lengthtake_bits!rest_sIResult methodsmany1rest_s method on IResult returns the remaining &str inputunwrap_err and unwrap_inc methods on IResultnot! will peek at the input and return Done if the underlying parser returned Error or Incomplete, without consuming the inputle_f32 and le_f64 parse little endian floating point numbers (IEEE 754)take_bits! is now more precisemany1 inccorectly used the len function instead of input_lenrecognize! had an early return that is removed nowlength_bytes and some documentationcond_with_error! combinatormany0!separated_list! fixesre_bytes_ work on byte slicescond_with_error! works like cond! but will return None if the condition is false, and Some(value) if the underlying parser succeededfold_many0!, fold_many1! and fold_many_m_n! will take a parser, an initial value and a combining function, and fold over the successful applications of the parserlength_bytes! converts the result of its child parser to usizetake_till! now imports InputLength instead of assuming it's in scopeseparated_list! and separated_nonempty_list! will not consume the separator if there's no following successfully parsed valuemany0!take_until_s!nom::Err now implements std::error::Errorhex_u32 does not parses more than 8 chars nowtake_while! and take_while1! will not perturb the behaviour of recognize! anymoreIResultIResult and NeededIResult and Neededapply_rf is renamed to apply_m. This will not warrant a major version, since it is part missing from the methods feture added in the 1.2.0 releaseregexp_macros feature that used regex! to precompile regular expressions has been replaced by the normal regex engine combined with lazy_static&input[input.len()..].regexp_macros and no_std feature build again and are now tested with Travis CI&str parsing and method parsersIResultalt_complete! combinatorhex_digittake_until_and_consume_s! for consumption of string data until a tagnamed!. The error type can now be specifiedalt_complete! works like the alt! combinator, but tries the next branch if the current one returned Incomplete, instead of returning directlytuple! combinator takes a list of parsers as argument, and applies them serially on the input. If all of them are successful, it willr eturn a tuple accumulating all the values. This combinator will (hopefully) replace most uses of chain!method!, call_m! and apply_rf! combinatorspeek! compilation with bare functions&str parsers were splitting data at the byte level, not at the char level, which can result in inconsistencies in parsing UTF-8 characters. They now use character indexesIResult<I,O,E> (with specified error type instead of implicit)This release adds a lot of features related to &str parsing. The previous versions were focused on &[u8] and bit streams parsing, but there's a need for more text parsing with nom. The parsing functions like alpha, digit and others will now accept either a &[u8] or a &str, so there is no breaking change on that part.
There are also a few performance improvements and documentation fixes.
&str parsingOption and Vec importstag_s!, take_s!, is_a_s!, is_not_s!, take_while_s!, take_while1_s!, take_till_s!value! is a combinator that always returns the same value. If a child parser is passed as second argument, that value is returned when the child parser succeedstag! will now compare even on partial input. If it expects “abcd” but receives “ef”, it will now return an Error instead of Incompletemany0! and others will preallocate a larger vector to avoid some copies and reallocationsalpha, digit, alphanumeric, space and multispace now accept as input a &[u8] or a &str. Additionally, they return an error if they receive an empty inputtake_while!, take_while1!, take_while_s!, take_while1_s! wilreturn an error on empty inputmany0! or many1! returns Incomplete, it will return Incomplete too, possibly updating the needed sizeOption, Some, None and Vec are now used with full path importsThis releases makes the 1.0 version compatible with Rust 1.2 and 1.3
Stable release for nom. A lot of new features, a few breaking changes
length_bytes! combinatorIResult::Error can now use custom error types, and is generic over the input typenom::ErrorCode is now nom::ErrorKindfilter! has been renamed to take_while!chain! will count how much data is consumed and use that number to calculate how much data is needed if a parser returned Incompletealt! returns Incomplete if a child parser returned Incomplete, instead of skipping to the next parserIResult does not require a lifetime tag anymore, yay!complete! will return an error if the child parser returned Incompleteadd_error! will wrap an error, but allow backtrackinghex_u32 parserIncomplete is better for most parsers nowThis release fixes a few issues and stabilizes the code.
tap! and extending dbg! and dbg_dmp!str::from_utf8tap!, dbg! and dbg_dmp! now accept function parameterscount_fixed! must be Copychain! calculates how much data is needed if one of the parsers returns `Incompletechain! can return IncompleteConsidering the number of changes since the last release, this version can contain breaking changes, so the version number becomes 0.4.0. A lot of new features and performance improvements!
chain! and for the rest parsermany0! and many1!switch! combinator ideais_a!count_fixed! must take an explicit type as argument to generate the fixed-size arraychain!count! can take 0 elementsis_a! and is_not! can now consume the whole inputMemProduceropt! returns Done(input, None) if the child parser returned Incomplete`rest will return the remaining inputswitch! applies a first parser then matches on its result to choose the next parsertake_till!, take_while! and take_while1!alt! can return Incompletefilter!count!: a Vec of the right size is directly allocatedbits! for bit level parsing. It indicates that all child parsers will take a (&[u8], usize)as input, with the second parameter indicating the bit offset in the first byte. This allows viewing a byte slice as a bit stream. Most combinators can be used directly under bits!take_bits! takes an integer type and a number of bits, consumes that number of bits and updates the offset, possibly by crossing byte boundariessrc/bits.rs. This applies to tag!, is_not!, is_a!, filter!, take!, take_str!, take_until_and_consume!, take_until!, take_until_either_and_consume!, take_until_either!`filter!opt_res! applies a parser and transform its result in a Result. This parser never failscond_reduce! takes an expression as parameter, applies the parser if the expression is true, and returns an error if the expression is falsetap! pass the result of a parser to a block to manipulate it, but do not affect the parser's resultAccReader is a Read+BufRead that supports data accumulation and partial consumption. The consume method must be called afterwardsto indicate how much was consumedu16!, u32!, u64!, i16!, i32!, i64! take an expression as parameter, if the expression is true, apply the big endian integer parser, if false, the little endian versionmap_opt! and map_res! had issues with argument order due to bad macrosdelimited! did not compile for certain combinations of argumentsfilter! did not return a byte slice but a fixed arrayStepper: wrap a Producer, and call the method step with a parser. This method will buffer data if there is not enough, apply the parser if there is, and keep the rest of the input in memory for the next callReadProducer: takes something implementing Read, and makes a Producer out of itseparated_pair! and delimited! did not work because an implementation macro was not exportedMemProducer reached its end, it should always return Eofmap! had issues with argument matchingexpr_res! and expr_opt! evaluate an expression returning a Result or Opt and convert it to IResultAsBytes is implemented for fixed size arrays. This allows tag!([41u8, 42u8])count_fixed! argument parsing works againfailed(&self, error_code) method in case of parsing errornamed! now handles the alternative named!(pub fun_name<OutputType>, ...)filter! now returns the whole input if the filter function never returned falsetake! casts its argument as usize, so it can accepts any integer type nowcount_fixed! returns a fixed arraycount! is back to the previous behaviour, returning a Vec for sizes known at runtimenom::util are now directly in nom::cond!chain!chain! can now have mutable fieldscond! had an infinite macro recursionchain! generates less code now. No apprent compilation time improvementcount!le_i8, le_i16, le_i32, le_i64: little endian signed integer parsersalt! parser compiles much faster, even with more than 8 branchescount! can now return a fixed size array instead of a growable vectortake_str parser and the function application combinatortake_str!: takes the specified number of bytes and return a UTF-8 stringapply!: do partial application on the parameters of a functionNeeded::Size now contains a usize instead of a u32be_i8, be_i16, be_i32, be_i64: big endian signed integer parserscore feature can be passed to cargo to build with no_stderror! will cut backtracking and return directly from the parser, with a specified error codeeof parser, successful if there is no more inputFlatMap, FlatpMapOpt and Functor traits (replaced by map!, map_opt! and map_res!)named! can now declare public functions like this: named!(pub tst, tag!("abcd"));pair!(X,Y) returns a tuple (x, y)separated_pair!(X, sep, Y) returns a tuple (x, y)preceded!(opening, X) returns xterminated!(X, closing) returns xdelimited(opening, X, closing) returns xseparated_list(sep, X) returns a Vec<X>separated_nonempty_list(sep, X) returns a Vec<X> of at list one elementmany0! and many1! forbid parsers that do not consume inputis_a!, is_not!, alpha, digit, space, multispace will now return an error if they do not consume at least one bytecount! to apply a parser a specified number of timescond! applies a parser if the condition is metutil::*opt! would not compilecollectionsworks with rustc 1.0.0-dev (81e2396c7 2015-03-19) (built 2015-03-19)
Incomplete(_), but if more granularity is mandatory, Needed can be matched tooalt! can pass the result of the parser to a closuretake_* macros changed behaviour, the default case is now not to consume the separator. The macros have been renamed as follows: take_until! -> take_until_and_consume!, take_until_and_leave! -> take_until!, take_until_either_and_leave! -> take_until_either!, take_until_either! -> take_until_either_and_consume!peek! macro: matches the future input but does not consume itlength_value! macro: the first argument is a parser returning a n that can cast to usize, then applies the second parser n times. The macro has a variant with a third argument indicating the expected input size for the second parsernamed! macro is provided to create functions from parsers. Please be aware that nesting parsers comes with a small cost of compilation time, negligible in most cases, but can quickly get to the minutes scale if not careful. If this happens, separate your parsers in multiple subfunctions.named!, closure! and call! macros used to support the unnamed syntaxmap!, map_opt! and map_res! to combine a parser with a normal function, transforming the input directly, or returning an Option or Resultis_a! is now working properlyo! macro does less than chain!, so it has been removedfold0! and fold1! macros were too complex and awkward to use, the many* combinators will be useful for most uses for now