Rob Reid

Anyone who ends up writing a load tester will soon arrive at the need for something to execute a function a number of times per second for a number of seconds… In the wonderful load testing CLI tool hey, JBD accomplishes this elegantly with channels, optionally configuring and waiting on a ticker channel:

var throttle <-chan time.Time
if b.QPS > 0 {
	throttle = time.Tick(time.Duration(1e6/(b.QPS)) * time.Microsecond)
}

...

if b.QPS > 0 {
	<-throttle
}

Here’s some code that demonstrates the throttling acheived with this code. It will run 10 queries per second, waiting inside the for loop if a value greater than zero is provided. If no value is provided for queries per second, the application will run as fast as possible:

package main

import (
	"fmt"
	"time"
)

func main() {
	qps := 5

	var throttle <-chan time.Time
	if qps > 0 {
		throttle = time.Tick(time.Duration(1e6/(qps)) * time.Microsecond)
	}

	for i := 0; i < 10; i++ {
		if qps > 0 {
			<-throttle
		}
		fmt.Println(time.Now().Format(time.RFC3339))
	}
}

When run, this code will output something similar to the following, note that only 5 things are printed in any 1 second period:

2019-05-28T14:44:04+03:00
2019-05-28T14:44:04+03:00
2019-05-28T14:44:04+03:00
2019-05-28T14:44:04+03:00
2019-05-28T14:44:05+03:00
2019-05-28T14:44:05+03:00
2019-05-28T14:44:05+03:00
2019-05-28T14:44:05+03:00
2019-05-28T14:44:05+03:00
2019-05-28T14:44:06+03:00

throttle

To save future headaches, copy pasta, and codebase crawls, I’ve created a package that reduces the amount of manual wait code required.

// Create a throttle to run something 5 times per second:
t := throttle.New(5, time.Second)

Once a throttle has been created, choose one of the following methods depending on your use case, both will operate at a resolution of 5 operations per second, as configured in the throttle:

t.Do(context.Background(), total int, f func()) - Run a total number of queries. Useful if you know how many requests you’d like to run ahead of time.

t.DoFor(context.Background(), d time.Duration, f func()) - Run for a given duration. Useful if you know how long you’d like to run for but don’t know how many requests in total.

Under the hood, a simple function calculates what “30 requests per minute” means as a portion of the resolution you’ve provided and returns it as something to be used to feed the throttle:

func qos(rate int64, res time.Duration) time.Duration {
	micros := res.Nanoseconds()
	return time.Duration(micros/rate) * time.Nanosecond
}

For example:

qos(1, time.Hour)           // 1h0m0s
qos(1, time.Minute)         // 1m0s
qos(1, time.Second)         // 1s
qos(1, time.Millisecond)    // 1ms
qos(1, time.Microsecond)    // 1µs
qos(60, time.Hour)          // 1m0s
qos(60, time.Minute)        // 1s
qos(1000, time.Second)      // 1ms
qos(1000, time.Millisecond) // 1µs
qos(1000, time.Microsecond) // 1ns

If you need to cancel the Do function at any time, pass a cancellable context.Context as follows:

ctx, cancel := context.WithCancel(context.Background())
defer cancel()

t.Do(ctx...

Examples

The following example demonstrates a throttle configured with a resolution of 5 requests per second, running for 10 operations, as in the first example:

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/codingconcepts/throttle"
)

func main() {
	t := throttle.New(5, time.Second)
	t.Do(context.Background(), 10, func() {
		fmt.Println(time.Now().Format(time.RFC3339))
	})
}

The output confirms that 10 operations were invoked, with no more than 5 things being printed at any one time:

2019-05-28T14:48:19+03:00
2019-05-28T14:48:20+03:00
2019-05-28T14:48:20+03:00
2019-05-28T14:48:20+03:00
2019-05-28T14:48:20+03:00
2019-05-28T14:48:20+03:00
2019-05-28T14:48:21+03:00
2019-05-28T14:48:21+03:00
2019-05-28T14:48:21+03:00
2019-05-28T14:48:21+03:00

The following example demonstrates a throttle configured with a resolution of 5 requests per second, running for 2 seconds:

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/codingconcepts/throttle"
)

func main() {
	t := throttle.New(5, time.Second)
	t.DoFor(context.Background(), time.Second*2, func() {
		fmt.Println(time.Now().Format(time.RFC3339))
	})
}

Once again, we can see that the function has been invoked 10 times:

2019-05-28T14:56:41+03:00
2019-05-28T14:56:41+03:00
2019-05-28T14:56:41+03:00
2019-05-28T14:56:41+03:00
2019-05-28T14:56:42+03:00
2019-05-28T14:56:42+03:00
2019-05-28T14:56:42+03:00
2019-05-28T14:56:42+03:00
2019-05-28T14:56:42+03:00
2019-05-28T14:56:43+03:00