Index by Regex?

[bluss]

It’s a cool hack, not sure if there is much more to say than that

[ArtemGr]

Wait, aren’t regexes unicode-aware? When I worked with regexes in Perl, C++ (PCRE, re2) and Java they were. If they’re unicode-aware, it should be absolutely no problem to index a string by regex because the index will always return a slice to a correct UTF-8 sequence.

[ArtemGr]

I’d like to see it bundled with https://crates.io/crates/regex

[DanielFath]

That question makes no sense, if you know Unicode supports hieroglyphs and ancient Sumerian letters you’ll realize why that’s nearly impossible.

How do you do Regex, which returns A where B is below it and C bottom left corner? Regex only works ok things that are written left to right. If I write ab* and use a RTL(right to left) language it should match bbbbba!

You could limit Regex only to European-like LTR languages.

[Florob]

You’re conflating (matching on) display with matching a stream of copdeoints. There is no reason regex shouldn’t work on Unicode strings. In particular matching on Unicode Category is supported by the regex crate. You are right though that writing regexes in the presence of RTL characters can be confusing. It is likely better to use an escape sequence in those cases. Depending on ones needs one might also have to think about Unicode Normalization, but none of that fundamentally prevents regex from working.

[reem]

Since regex is no longer a part of the stabilizing distribution, I don’t see why it shouldn’t support more ambitious and experimental APIs like the one suggested here.

regex is a pre-1.0 library; if these features prove undesirable, we can just remove them.

[ArtemGr]

It’s one of those features which are very unlikely to get much attention unless advertized by including them in the existing library. I never knew that Ruby had such a feature, for example, and wouldn’t expect it to be present in any other language or library I’m working with. Even in Ruby, I doubt a lot of people aware of it and use it. Still, it’s a neat feature to have.

[DanielFath]

Well, often regex try to capture some combination of grapheme and not codepoints. My experience is that people don’t think in term of codepoints, they think more in terms of XX.XX.XXXX that matches visual appearance.

[burntsushi]

I’m honestly not a huge fan. It’s a nice trick, but it encourages task failure whenever the regex fails to match.

If you really want the indexing, I see no reason why this has to be in the regex crate. A wrapper type is a not a burden here, and it could even be less typing. (Just create your own constructor that calls Regex:new.)

Also, I think we should be moving toward stabilizing the regex crate. It’s rather popular, so we’ll risk making it de facto stable if we wait too long.

[kstep]

Yes, I get your POV. You are basically right about panics. In Ruby when regex “indexing” fails, it just returns nil, so it’s OK to have such sugar, but in Rust it’s not that simple. Now I think this feature doesn’t fit Rust ideology after all. And I don’t think a crate with such feature will be used by any significant number of users, so it just doesn’t worth efforts to create it.

[ArtemGr]

In Ruby when regex “indexing” fails, it just returns nil, so it’s OK to have such sugar, but in Rust it’s not that simple.

In Rust that would be Option<something>.

Now I think this feature doesn’t fit Rust ideology after all.

Right now Rust is in it’s “be minimalistic before 1.0” stage, everyone’s holding they breath waiting for Rust to become stable, but I’ve heard voices that Rust will start to grow some extra weight later.

Writing

if let Some (match) = a_string[regex!(r"...")] {}

seems useful to me, it’s easier to remember than the names of the methods regex crate uses (I keep hitting the docs to check them).

I’d vouch for making a crate even if there would be a few users. At least the gist will be easy to find that way.

[kstep]

You can’t do return Option<something> from index. See the signatures (simplified a little):

trait Index<Idx> {
  type Output;
  fn index(&'a self, index: &Idx) -> &'a Output;
}

You have to return a reference to something, no owned data can be returned from index(). And the reference inherently must to be something from self (due to lifetime restrictions).

[burntsushi]

In Rust that would be Option<something>.

Right, but when you’re indexing, the regex match has to be unwrapped which will cause task failure if the match fails. I suppose an alternative to this is to return a zero length slice if the regex doesn’t match. (This will conflate zero-length matches with non-matches, but that’s probably OK to gloss over for a convenience such as indexing.)

seems useful to me, it’s easier to remember than the names of the methods regex crate uses (I keep hitting the docs to check them).

I agree it’s a neat trick, I’m just trying to say that this can be done entirely outside of the regex crate with almost no downsides. (Wrapper types are usually inconvenient because you have to re-implement all of the behavior of the underlying type, but in this case, all you really need it for is construction.)

Also, more importantly, I’m pretty sure that your if let Some(match) = a_string[regex!(r"...")] {} construct is not possible. (Whoops, @kstep beat me to it.)

[kstep]

I suppose an alternative to this is to return a zero length slice if the regex doesn’t match.

Hmm, that’s not that elegant, but it may work. Not sure about ergonomics of such implementation, but I think it worth to make a PoC crate and give it a try in real world.

[ArtemGr]

Ahh. Too bad one can’t get an Option from an index. That makes it less useful than I thought.

[kstep]

I just published simple PoC crate regindex (repository).

[ArtemGr]

Apparently one can’t make it to work as transparent as in Ruby (e.g. without the ReIdx wrapper) without it being a part of the regex crate?

[burntsushi]

My suggestions to make this more ergonomic outside of the regex crate seem to have gotten lost in the noise. I filed an issue with suggestions: https://github.com/kstep/regindex/issues/1

[ArtemGr]

If we’re going so far as to define a new macro, rei!, then why bother and not define matches! ("hello", r"el"), returning an Option to boot. So far I feel this attempt falls short of the Ruby version.

[burntsushi]

I’m here to help you make the best with the tools that we have. I’m not a fan of indexing at all (I can’t imagine when I would use it in real code because it encourages task failure when the regex doesn’t match), but I was trying to work within the parameters of the OP. Certainly, some other macro that doesn’t use indexing at all might be more convenient, although I’m not convinced that it would belong in the regex crate proper.