Allowing flags anywhere on the CLI¶
Writing CLIs in Go is great, but there's one annoyance with the stdlib flag package:
flags must be defined before any positional arguments.
For example, if you have a --debug flag, you can't use it after a positional argument, it'll be
treated as a 2nd positional argument.
Flag parsing stops just before the first non-flag argument (- is a non-flag argument) or after
the terminator --. 1
The Go project took a firm stance on this issue. For those interested, see the following GitHub issues (there are more, but these are the most relevant):
The challenge is that most users nowadays expect flag order to be irrelevant, especially since tools from other ecosystems behave this way.
The Command Line Interface Guidelines outlines best practices for authoring CLI tools — and they're pretty good. Of note, this snippet:
The order of flags, generally speaking, does not affect program semantics.
The Heroku CLI Style Guide also
encourages this approach. Only mentioning it because of nostalgia, the heroku CLI had a fairly
nice user experience back in the day.
This also allows the user to specify the flags in any order
There's loads more examples, guides and popular tools I could link to, but we'll leave it at that.
So what?¶
Imagine you're tweaking a (long) command and something isn't working as expected. You reach for a
global --verbose or --debug flag, but now you have the cognitive overhead of figuring out where
to slot the flag.
Or, you're a new user who just installed a shiny new CLI written in Go and it's not immediately working as you'd expect. You're not sure if you're missing a flag, or if you're using it wrong. Leading to a furstating experience.
Weak arguments? Maybe. But it bugs me, and I'm sure I'm not alone.
There's a wealth of third-party Go packages and entire CLI frameworks that handle this, but I want to do the bare minimum to get things working for 99% of cases, using only the stdlib when possible.
So here's a fairly simple solution that works for me. There's one edge case and one use of unsafe,
but overall, I think it does the job right. Gotcha section for more details.
Snippet¶
Copied from mfridman/xflag.
func ParseToEnd(f *flag.FlagSet, arguments []string) error {
if err := f.Parse(arguments); err != nil {
return err
}
if f.NArg() == 0 {
return nil
}
var args []string
remainingArgs := f.Args()
for i := 0; i < len(remainingArgs); i++ {
arg := remainingArgs[i]
if len(arg) > 1 && arg[0] == '-' {
if arg == "--" {
args = append(args, remainingArgs[i:]...)
break
}
if err := f.Parse(remainingArgs[i:]); err != nil {
return err
}
remainingArgs = f.Args()
i = -1
continue
}
args = append(args, arg)
}
if len(args) > 0 {
argsField := reflect.ValueOf(f).Elem().FieldByName("args")
argsPtr := (*[]string)(unsafe.Pointer(argsField.UnsafeAddr()))
*argsPtr = args
}
return nil
}
// ParseToEnd is a drop-in replacement for flag.Parse. It improves upon the standard behavior by
// parsing flags even when they are interspersed with positional arguments. This overcomes Go's
// default limitation of stopping flag parsing upon encountering the first positional argument. For
// more details, see:
//
// - https://github.com/golang/go/issues/4513
// - https://github.com/golang/go/issues/63138
//
// This is a bit unforunate, but most users nowadays consuming CLI tools expect this behavior.
func ParseToEnd(f *flag.FlagSet, arguments []string) error {
if err := f.Parse(arguments); err != nil {
return err
}
if f.NArg() == 0 {
return nil
}
var args []string
remainingArgs := f.Args()
for i := 0; i < len(remainingArgs); i++ {
arg := remainingArgs[i]
// If the arg looks like a flag, parses like a flag, and quacks like a flag, then it
// probably is a flag.
//
// Note, there's an edge cases here which we EXPLICITLY do not handle, and quite honestly
// 99.999% of the time you wouldn't build a CLI with this behavior.
//
// If you want to treat an unknown flag as a positional argument. For example:
//
// $ ./cmd --valid=true arg1 --unknown-flag=foo arg2
//
// Right now, this will trigger an error. But *some* users might want that unknown flag to
// be treated as a positional argument. It's trivial to add this behavior, by using VisitAll
// to iterate over all defined flags (regardless if they are set), and then checking if the
// flag is in the map of known flags.
if len(arg) > 1 && arg[0] == '-' {
// If we encounter a "--", treat all subsequent arguments as positional.
if arg == "--" {
args = append(args, remainingArgs[i:]...)
break
}
if err := f.Parse(remainingArgs[i:]); err != nil {
return err
}
remainingArgs = f.Args()
i = -1 // Reset to handle newly parsed arguments.
continue
}
args = append(args, arg)
}
if len(args) > 0 {
// Access the unexported 'args' field in FlagSet using reflection and unsafe pointers.
argsField := reflect.ValueOf(f).Elem().FieldByName("args")
argsPtr := (*[]string)(unsafe.Pointer(argsField.UnsafeAddr()))
*argsPtr = args
}
return nil
}
Gotcha¶
Disclaimer, there's 2 things to be aware of:
-
It uses the
unsafepackage. Theargsfield of aflag.FlagSetis unexported and there's no setter, so we need to use reflection and unsafe pointers to access it. Best case, the field name never changes, but there's never 100% guarantee with unexported fields. -
Some CLI author might want to treat an unknown flag as a positional argument. For example:
The use case here is treating --unknown-flag=foo as a positional argument. It's fairly easy to
solve, but I'm not sure if it's worth the effort.
If it looks like a flag, parses like a flag, and quacks like a flag, then it probably is a flag.