This library is released every month, usually at the first week of month.
Changelog in 2022. This is changelog for share module since v0.33.0 until v0.42.0.
Changelog in 2021. This is changelog for share module since v0.22.0 until v0.32.0.
Changelog in 2020. This is changelog for share module since v0.12.0 until v0.21.0.
Changelog from 2018 to 2019. This is changelog for share module since v0.1.0 until v0.11.0.
This release bring many enhancements to lib/websocket including timeout, handling upgrade and read/write concurrently using goroutine.
- lib/net: changes the WaitRead/Event model on Poll
-
Previously, the Pool’s WaitRead and WaitReadEVent methods return list of file descriptor (fd) and keeps the fd in the pool. In case we want to process the returned fd concurrently, by running it in different goroutine, the next call WaitRead may return the same fd if its goroutine not fast enough to read from fd.
This changes fix this issue by removing list of fd from poll and set the fd flag to blocking mode again after returning it from WaitRead or WaitReadEvent.
This changes also remove the ReregisterRead and ReregisterEvent methods since it is not applicable anymore.
- lib/websocket: call Quit when handshake contains close or invalid frame
-
If the HTTP handshake response contains trailing frame, handle it directly. If the frame is invalid or contains control close operation, call Quit directly to trigger the HandleQuit if its defined by user.
- lib/websocket: revert maxBuffer back to 1024
-
In v0.47.0 we increase the maxBuffer to 4096 to try increasing the performance when handling large payload. Turns out increasing this break the autobahn test suite.
- lib/ascii: add type Set
-
The Set type is a bitmap that represent list of ASCII characters for faster lookup.
A Set is a 36-byte value, where each bit in the first 32-bytes represents the presence of a given ASCII character in the set. The remaining 4-bytes is a counter for the number of ASCII characters in the set. The 128-bits of the first 16 bytes, starting with the least-significant bit of the lowest word to the most-significant bit of the highest word, map to the full range of all 128 ASCII characters. The 128-bits of the next 16 bytes will be zeroed, ensuring that any non-ASCII character will be reported as not in the set.
- lib/net: implement generic PollEvent
-
The PollEvent contains file descriptor and the underlying event based on OS, unix.EpollEvent on Linux or unix.Kevent_t on BSD.
The Poll interface provides two APIs to works with PollEvent, WaitReadEvents that return list of PollEvent ready for read, and ReregisterEvent to register the event back to poll (only for Linux).
- lib/websocket: add option to set read/write timeout on Server
-
The ReadWriteTimeout define the maximum duration the server wait when receiving/sending packet from/to client before considering the connection as broken.
Default read-write timeout is 30 seconds if not set.
This changes affect the exported function Send and Recv by adding additional parameter timeout to both of them.
- lib/websocket: handle concurrent upgrade using goroutine
-
The maxGoroutineUpgrader define maximum goroutines running at the same time to handle client upgrade. The new goroutine only dispatched when others are full, so it will run incrementally not all at once. Default to defServerMaxGoroutineUpgrader (128) if its not set.
- lib/websocket: handle concurrent Server read using goroutines
-
The Server now dispatch a goroutine to consume event from poll reader for each client connection that is ready to read. The maximum number of goroutine is defined in ServerOptions maxGoroutineReader, which currently set to 1024.
- lib/websocket: handle concurrent ping using goroutines
-
The maximum goroutines is quarter of max queue. The new goroutine for pinger will be dispatched when no goroutine can consume the current processed connection.
- websocket/testdata: rewrite autobahn test using container
-
Since the autobahn script can only run on Python 2, it become hard to setup and run the test on distro that does not provide Python 2 anymore. The autobahn repository recommend to use docker instead.
When testing the server, we simplify it by using make task "test-server". The test-server task run our test server in background, and then run the autobahn fuzzingclient from container. Once the tests completed, we trigger the server to shutdown by sending text frame with payload "shutdown".
When testing the client, we simplify it by using make task "test-client". The test-client task run the autobahn fuzzingserver and then we run our client. Once client finished, we trigger the server to generate the reports and cleanup the container.
- email/maildir: major refactoring plus adding unit tests
-
This changes remove all unneeded methods from Manager leave it with four methods: Delete, FetchNew, Incoming, OutgoingQueue.
Also, we add the type filename to generate file name for tmp and new directory.
- lib/email: unexport the field ContentType in the field
-
The field ContentType will be set only when the field Name is "Content-Type" so it’s not always exist on each field. To get the field ContentType, use Header.ContentType().
- lib/dns: fix zone parsing on SOA record with single line
-
Due to refactoring in c376eccd25, parsing SOA record with single line return an error: "parseSOA: line 2: incomplete SOA statement '0'".
- lib/memfs: ignore permission error when scanning directory content
-
Instead of returning error, skip the directory that we cannot read and continue to process the other.
- lib/memfs: fix panic when watched file deleted or renamed
-
When the file being watched is deleted, sometimes it will cause panic.
- lib/email: fix parsing multiple parameters in ContentType
-
While at it, also fix the ContentType String method to prefix ';' before appending parameter key and value.
- cmd/bcrypt: CLI to compare or generate hash using bcrypt
-
The bcrypt command has two subcommand "compare" and "gen". The "compare" subcommand accept two parameter the hash and plain text. The "gen" subcommand accept only one parameter, the plain text to be hashed.
- lib/sql: add type DmlKind
-
The DmlKind define the kind for Data Manipulation Language.
- email/maildir: implement Folder
-
Folder is a directory under maildir that store messages per file. A folder contains three directories: tmp, new, and cur; and an empty file "maildirfolder".
- lib/net: add function WaitAlive
-
WaitAlive try to connect to network at address until timeout reached. If connection cannot established it will return an error.
Unlike [net.DialTimeout], this function will retry not returning an error immediately if the address has not ready yet.
- lib/smtp: implement Client SendEmail
-
Somehow in 3a1a2715b25f, we include this method without implementing it.
The SendEmail method simplify sending email by automatically create [MailTx] for passing it to method Client.MailTx.
The test right now use live connection since the Server is not ready yet.
- lib/dns: add option to set debug level in ServerOptions
-
This options replace the global debug package.
- lib/dns: do not cache empty answers
-
The use case if one use and switch between two different networks with internal zone, frequently. For example, if on network Y they have domain MY.Y and current connection is X, request to MY.Y will return an empty answers. Once they connect to Y again, any request to MY.Y will not be possible because rescached caches contains empty answer for MY.Y.
- _bin/go-test-coverhtml: add parameter to run specific test
-
The second parameter is optional. It is passed to -run= argument in "go test". Default value is ".", or all functions.
- lib/http: redirect path with slash if request is directory
-
If request path is a directory and it is not end with slash, redirect request to location with slash to allow relative links works inside the HTML content.
For example, a "/page/index.html" contains links href="sub.html" (where "sub.html" is inside "/page" directory). If request to "/page" (without end with slash) return content of "/page/index.html", then when user click on sub.html it will request to "/sub.html" instead of "/page/sub.html".
- lib/email: handle obsolete white spaces and comment when unpacking date
-
In the obsolete syntax, white space and comments can appear between many more element, for example the folloing Date value are valid
Date : Fri, 21 Nov 1997 09(comment): 55 : 06 -0600
This changes handle this by sanitizing the Field value, removing comment and merge multiple spaces into one, before parsing it.
- lib/email: set the Field Type and unpack its value on ParseField
-
Once the field Name has detected and its Value is valid, we can unpack the Value based to type that it represent, for example to Date or Mailbox.
This changes remove calling to unpack directly in some tests and check an error when testing ParseHeader.
- lib/net: increase the maximum poll events
-
The maxQueue define the number of events that can be read from poll at one time. Using 128 seems to small for high throughput networks. Increasing this number also increase the memory consumed by process. Maybe later we can export this function as option when creating poll.
- lib/websocket: increase the max buffer and queue for better throughput
-
The maxBuffer increased from 1024 to 4096 bytes. The reason that we use 1024 previously is related to MTU size and maximum payload in TCP (although its higher, 1460 bytes).
The maxQueue increase from 128 to 4096.
- all: remove any usage of debug.Value in all packages
-
Using global debug value for all packages turns out is not a good idea.
- lib/test: update documentation related to Assert and Data
-
The documentation is based on the article published at https://kilabit.local/journal/2023/go_test_data/ after reviewing and explain how to use both of them to public.
- all: record the contributors of this module in file AUTHORS
-
== share v0.46.0 (2023-05-02)
This release deprecated lib/io and lib/parser.
- lib/reflect: remove the third return value from Marshal
-
The third, boolean, return value is redundant with the second error value.
- lib/bytes: changes the DumpPrettyTable output format
-
The change is to accommodate large bytes data, more than 0xFFFF. The hex address in the first column is increased to 8 digits, the characters compacted without space in between.
- lib/os: merge some functions from lib/io
-
Functions like Copy, IsBinary, IsDirEmpty, IsFileExist, RmdirEmptyAll are read and operate on file and directory on operating system level, so it is not correct to put it in package io.
- lib/strings: merge lib/parser here
-
The first idea of parser is to provide generic parser for both bytes and string. After we introduce lib/parser there is not much changes to that package. Also, since we create another Parser in lib/bytes that accept and return token as []byte, the lib/parser is not unique anymore.
The following function/methods changes to minimize conflict in the future,
-
Lines become LinesOfFile
-
New become NewParser
-
Open become OpenForParser
-
Token become Read
-
TokenEscaped become ReadEscaped
-
TokenTrimSpace become ReadNoSpace
-
- lib/bytes: implement function ParseHexDump
-
The ParseHexDump parse the default output of [hexdump](1) utility from parameter in back into stream of byte.
An example of default output of hexdump is
0000000 7865 5f70 6964 2f72 0000 0000 0000 0000 0000010 0000 0000 0000 0000 0000 0000 0000 0000 * 0000060 0000 0000 3030 3030 3537 0035 3030 3130
The first column is the address and the rest of the column is the data. Each data column is 16-bit words in big-endian order, so in the above example, the first byte would be 65, second byte is 78 and so on. The asterisk "*" means that the address from 0000020 to 0000050 is equal to the previous line, 0000010.
[hexdump]: https://man.archlinux.org/man/hexdump.1
- lib/bytes: implement tokenize Parser
-
The Parser type parse stream of byte using one or more delimiters as separator between token.
- lib/bytes: add function TrimNull
-
The TrimNull function remove 0 value ("\0" or NULL in C) at leading and trailing of input.
- lib/net: add method WriteTo to ResolvConf
-
The WriteTo method write the ResolvConf as text.
- lib/time: calculate the next event before notify the user on Scheduler run
-
This allow user to call the Next method, to know the next time the scheduler will be triggered, after receiving the event.
- lib/reflect: add option to skip processing struct field in Do/IsEqual
-
A struct’s field tagged with
noequal:"", its value will not be processed for equality.
- lib/reflect: use doEqual inside IsEqual
-
Previously, IsEqual internally use isEqual, which have the same logic as doEqual. This changes minimize duplicate code between IsEqual and DoEqual, by calling doEqual for both of functions.
- lib/time: replace lib/io#Reader with lib/bytes#Parser
- lib/smtp: replace lib/io#Reader with lib/bytes#Parser
- lib/dns: replace lib/io#Reader with lib/bytes#Parser
- lib/http: replace lib/io#Reader with lib/bytes#Parser
- lib/email: replace lib/io#Reader with lib/bytes#Parser
- email/dkim: replace lib/io#Reader with lib/bytes#Parser
- lib/hunspell: replace lib/io with lib/os
- lib/hunspell: replace lib/parser with lib/strings
- lib/http: replace lib/parser with lib/strings
- lib/bytes: copy TokenFind to internal/bytes#TokenFind
-
This is to prevent import cycle later when we use lib/test in bytes.
This release set the Go version to 1.19.
- lib/net: changes the PopulateQuery logic
-
Previously, PopulateQuery only add the passed dname if the number of dots is greater than 0. After inspecting the result from dig and getenv, the dots seems does not affect the query. For example, if we have A record for domain "kilabit", both of those tools query name "kilabit" without adding local domain or domain in search.
- _bin: add shell script go-test-lint.sh to run test and lint sequentially
-
The go-test-lint.sh run Go test and if its success it will run predefined linter, in the current directory.
Arg 1: the method or function to test, default to ".".
The linter program and its argument is derived from environment variable GO_LINT. If its empty, it will try the following linter in order: revive and then golangci-lint.
To add additional arguments to Go test set the environment variable GO_TEST_ARGS.
- lib/bytes: add function DumpPrettyTable
-
The DumpPrettyTable write each byte in slice data as hexadecimal, ASCII character, and integer with 8 columns width.
- lib/bytes: add function SplitEach
-
The SplitEach funciton split the slice of byte into n number of bytes. If n is less or equal than zero, it will return the data as chunks.
- lib/dns: add function ParseZone
-
The ParseZone parse the content of zone from raw bytes.
Now that we have ParseZone, all tests that use zoneParser now can be replaced using combination of test.Data and ParseZone.
- lib/dns: add method WriteTo to Zone
-
The WriteTo method write the zone as text into io.Writer.
The result of WriteTo will be different with original content of zone file, since it does not preserve comment and indentation.
- lib/http: add function to parse multipart Range response for Client
-
The ParseMultipartRange parse the multipart/byteranges body or response from HTTP Range request. Each Content-Range position and body part in the multipart will be stored under RangePosition.
- lib/http: add support for HTTP Range in Server
-
For HTTP Server using HandleFS, the Range request is handled automatically. For other HTTP server, user can use the HandleRange function.
The HandleRange function handle HTTP Range request using "bytes" unit. The body parameter contains the content of resource being requested that accept Seek method.
If the Request method is not GET, or no Range in header request it will return all the body RFC7233 S-3.1.
The contentType is optional, if its empty, it will detected by http.ResponseWriter during Write.
- lib/io: add method ReplaceAll on Reader
-
The ReplaceAll method behave like standard bytes.ReplaceAll but start from current index.
- lib/parser: add method TokenTrimSpace
-
The TokenTrimSpace read the next token until one of the delimiter found, with leading and trailing spaces are ignored.
- lib/parser: add method SetDelimiters
-
The SetDelimiters replace the current delimiters.
- lib/telemetry: package for collecting and forwarding metrics
-
Package telemetry is a library for collecting various Metric, for example from standard runtime/metrics, and send or write it to one or more Forwarder. Each Forwarder has capability to format the Metric before sending or writing it using Formatter.
- lib/dns: fix packing, parsing, and saving MINFO resource data
-
Even thought the MINFO record not formally obsolete, according to Wikipedia, we still need to support this for backward compatibility.
When packing the resource data length does not include total length. When parsing, the RMailBox and EmailBox should be added the origin suffix if its not end with dot. When saving, the origin should be trimmed from RMailBox and EmailBox.
- lib/dns: fix packing and unpacking resource record HINFO
-
The rdata for HINFO contains two character-strings: CPU and OS. Previously, we pack the rdata section sequentially, without adding length on each of them: <RDLEN><CPU><OS>. The correct pack format should <RDLEN><LENGTH><CPU><LENGTH><OS>.
- lib/dns: fix parsing SRV record from zone file
-
Previous parseSRV start by parsing the _Service from tok, but the actual value of parameter tok is the Priority.
This changes fix this and as testing we use the example from RFC 2782.
- lib/dns: allow parsing TXT rdata without quote in zone file
-
Previously, the zone only parsing TXT record with double quote since most of the example that we found during implementation all use double quote.
This changes allow non-double quoted text in zone file with consequence that any spaces will terminated the rdata immediately.
Fixes #6
- lib/dns: handle zone file with CRLF line ending
-
While at it, fix parsing multiline SOA record where closing parentheses end on next lines.
Fixes #6
- lib/test: simplify the string diff output from Assert
-
In the output, instead of using %q we replace it with %s, because printing string with double quote cause escaping and hard to read This change may cause difference in white spaces not showed in the terminal.
In the diff changes, only print the Old and New, without printing each chunk.
- lib/time: implement Scheduler
-
Scheduler is a timer that run periodically based on calendar or day time.
A schedule is divided into monthly, weekly, daily, hourly, and minutely. An empty schedule is equal to minutely, a schedule that run every minute.
- lib/time: add new type Clock
-
Clock represent 24 hours time with hour, minute, and second. An hour value is from 0 to 23, a minute value is from 0 to 59, and a second value is from 0 to 59.
- lib/clise: fix potential data race between Push and Slice
-
The data race may occur if Push is called, the .last field is incremented and at the same time an other goroutine call Slice that access the .last field.
- lib/memfs: minimize data race on DirWatcher
-
Calling DirWatcher Stop while the start method set dw.ticker can cause data race. This changes fix this issue.
- go.mod: update all dependencies
-
This update use "go get all" which resolve to semver for each dependencies.
- all: set the test timeout to 1m
-
Some flaky test, caused by waiting for channel, require waiting for 10m before it considered fail. This changes we cut the default timeout to 1 minute.
- all: access embedded field or methods using the type name
-
This is for clarity in the code, better to be explicit by typing where the field or methods come from.
- lib/email: add an example for Filter method on Header
- cmd: temporarily hide unused commands
-
The cart, cascaded-random-forest, lnsmote, random-forest, and smote are part of completing thesis and they never used anymore.
- AUR: add go-test-coverhtml.sh and go-bench.sh into package
-
While at it, changes the go-bench count to 10.
- lib/websocket: replace math/rand.Read with crypto/rand.Read
-
The math/rand will be deprecated in Go 1.20.
- lib/http: add function MarshalForm
-
The MarshalForm marshal struct fields tagged with
form:into url.Values.The rules for marshaling follow the same rules as in [UnmarshalForm].
It will return an error if the input is not pointer to or a struct.
- clise: implement io Closer, Writer, StringWriter, and ByteWriter
- clise: add method UnmarshalJSON
-
The UnmarshalJSON convert JSON array into Clise.