zsh/scwrypts
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

basic runner format; write a MAIN function

Browse Source 
introduce --verbosity flag rather than mixed logging settings; correct color misnaming to ANSI convention; added sanity-check; simplified hello-world; created FZF_USER_INPUT to replace the confusing FZF_HEAD and FZF_TAIL

swap INFO for DEBUG

v3-to-v4 upgrade docs

bring some much-needed tender love and care to the scwrypts runner

improved i/o handling on the run executable means this is no longer relevant

FINALLY fix the weird cases for zsh/read builtin (particularly around reading one character from tty/pipe/file); also gave a --force-user-input flag in case you want to require user input on a yn prompt

update ZLE plugin so it no more make errors

FZF_(HEAD|TAIL) refactor to FZF_USER_INPUT

plugins/kubectl migration from v3 to v4

plugins/ci migration from v3 to v4

refactor py/lib into python-scwrypts subproject

verbosity is stupid lets call it log-level

fix bug with virtualenv loading

mergedeep to slow so I made my options dict shallow

hokay first iteration of python-dudes is ready

circleci configuration for python builds

npm package for scwrypts

3.9.1

initial build/test steps for nodejs

go

go

ok

ok

fix output

ok

ok

finalize publish steps
This commit is contained in:
Wryn (yage) Wagner 2024-02-06 14:06:44 -07:00
parent a200c1eb22
commit fec8c5e560
169 changed files with 15694 additions and 1451 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

128
.circleci/config.yml Normal file
View File

@ -0,0 +1,128 @@
---
version: 2.1
orbs:
python: circleci/python@2.1.1
executors:
python:
docker:
- image: cimg/python:3.11
resource_class: small
nodejs:
docker:
- image: node:18
resource_class: medium
jobs:
python-test:
executor: python
working_directory: ~/scwrypts/py/lib
steps:
- checkout:
path: ~/scwrypts
- run:
name: pytest
command: |
: \
&& pip install . .[test] \
&& pytest \
;
python-publish:
executor: python
working_directory: ~/scwrypts/py/lib
steps:
- checkout:
path: ~/scwrypts
- python/dist
- run: pip install twine && twine upload dist/*
nodejs-test:
executor: nodejs
working_directory: ~/scwrypts/zx/lib
steps:
- checkout:
path: ~/scwrypts
- restore_cache:
name: restore pnpm cache
keys:
- pnpm-packages-{{ checksum "pnpm-lock.yaml" }}
- run:
name: pnpm install
command: |
corepack enable
corepack prepare pnpm@latest-8 --activate
pnpm config set store-dir .pnpm-store
pnpm install
- save_cache:
name: save pnpm cache
key: pnpm-packages-{{ checksum "pnpm-lock.yaml" }}
paths:
- .pnpm-store
- run: pnpm test
- run: pnpm lint
- run: pnpm build
nodejs-publish:
executor: nodejs
working_directory: ~/scwrypts/zx/lib
steps:
- checkout:
path: ~/scwrypts
- restore_cache:
name: restore pnpm cache
keys:
- pnpm-packages-{{ checksum "pnpm-lock.yaml" }}
- run:
name: pnpm install
command: |
corepack enable
corepack prepare pnpm@latest-8 --activate
pnpm config set store-dir .pnpm-store
pnpm install
- save_cache:
name: save pnpm cache
key: pnpm-packages-{{ checksum "pnpm-lock.yaml" }}
paths:
- .pnpm-store
- run:
name: publish
command: |
: \
&& pnpm build \
&& pnpm version $(git describe --tags) \
&& pnpm set //registry.npmjs.org/:_authToken=$NPM_TOKEN \
&& pnpm publish --no-git-checks \
;
workflows:
python:
jobs:
- python-test
- python-publish:
requires: [python-test]
context: [pypi-yage]
filters:
branches:
only: main
nodejs:
jobs:
- nodejs-test
- nodejs-publish:
requires: [nodejs-test]
context: [npm-wrynegade]
filters:
branches:
only: main

3
README.md
View File

@ -6,6 +6,9 @@ In modern developer / dev-ops workflows, scripts require a complex configuration
Without a better solution, the developer is cursed to copy lines-upon-lines of variables into terminals, create random text artifacts, or maybe even commit secure credentials into source. Without a better solution, the developer is cursed to copy lines-upon-lines of variables into terminals, create random text artifacts, or maybe even commit secure credentials into source.
Scwrypts leverages ZSH to give hot-key access to run scripts in such environments. Scwrypts leverages ZSH to give hot-key access to run scripts in such environments.
## Major Version Upgrade Notice
Please refer to [Version 3 to Version 4 Upgrade Path](./docs/upgrade/v3-to-v4.md) when upgrading from scwrypts v3 to scwrypts v4!
## Dependencies ## Dependencies
Due to the wide variety of resources used by scripting libraries, the user is expected to manually resolve dependencies. Due to the wide variety of resources used by scripting libraries, the user is expected to manually resolve dependencies.

254
docs/upgrade/v3-to-v4.md Normal file
View File

@ -0,0 +1,254 @@
# Scwrypts Upgrade v3 to v4 Notes
Scwrypts v4 brings a big update to the *runstring for `zsh`-type scwrypts*.
I've found some of the boilerplate required by each individual script to be confusing and terse, and the goal here is to make it easier and safer to write and run scwrypts in this critical format.
Jump to [Technical Bits](#technical-bits) if you just want to get started with migration steps.
The actual migration _should take less than a minute per script_.
This document is deliberately verbose for future reference when I don't remember how any of this works.
## Ideology and History
Originally (scwrypts v2 and below) wanted to preserve the direct-use of individual scwrypts.
In those versions, executable files could be executed directly (outside of the `scwrypts` function) and still operate with minimal, unwanted consequences.
This resulted in a rigid structure which made code-sharing difficult at small scales and untenable in many non-trivial cases.
Scwrypts v3, begrudgingly introduced a pseudo-import syntax with `use`.
This sought to combat the issues of code-sharing and open up the structure of executable scwrypts to the scwrypts-writer.
Beyond just clarity, this allowed external libraries to be written and cross-referenced.
Although "importing" is an odd (anti?)feature to shell scripting, the way libraries could be written and reused was too helpful and I succumbed to write the `import.driver.zsh` module.
Throughout v3, I tried to maintain the "executability" of individual scwrypts.
It's ugly though.
Every individual scwrypt relies on `import.driver.zsh` and the context set up by the `scwrypts` executable.
While you _could_ run the executable file directly, it would misbehave at-best and fail pretty reliably.
So... here's v4!
Scwrypts v4 accepts the reality that, although `zsh` scwrypts are zsh, they do not stand alone without the proper context setup provided by `scwrypts`.
To improve usability, I've abstracted much of the boilerplate so you never have to see it.
I've injected safety mechanisms like `--help` arguments and utility mechanisms like flag separation (`-abc` is really `-a -b -c`) into all v4 zsh scwrypts.
You don't have to worry about checking the context, v4 does that for you!
You don't have to worry about execution, v4 does that for you!
So!
Are you coupling your zsh scripts to `scwrypts` when you write them? Yes.
Is that a bad thing? I don't think so.
Shell-scripting is such a critical coupler to real-life systems.
High-risk-high-impact to SLAs means we cannot allow context mistakes by sysadmins and users.
Reusability between local machine, cloud runtime, and CI pipelines is a must.
And if you have a need to reign all that in to single, isolated executable files...
...then good luck <3
## Technical Bits
Big idea: let's get rid of v3 boilerplate and make things easy.
### Your executable must be in a MAIN function
A main function in shell scripts?
Weird!
Don't worry, it's easy.
Take your original scwrypt, and slap the executable stuff into a function called `MAIN` (yes, it must be _exactly_, all-caps `MAIN`):
```diff
#!/bin/zsh
#####################################################################
DEPENDENCIES+=(dep-function-a dep-function-b)
REQUIRED_ENV+=()
use do/awesome/stuff --group my-custom-library
CHECK_ENVIRONMENT
#####################################################################
- echo "do some stuff here"
- # ... etc ...
- SUCCESS "completed the stuff"
+ MAIN() {
+ echo "do some stuff here"
+ # ... etc ...
+ SUCCESS "completed the stuff
+ }
```
**Don't invoke the function!**
Scwrypts will now do that on your behalf.
I've already written many scwrypts which _almost_ used this syntax.
All I had to do in this case was delete the function invocation at the end:
```diff
# ... top boilerplate ...
MAIN() {
SUCCESS "look at me I'm so cool I already wrote this in a main function"
}
-
- #####################################################################
- MAIN $@
```
Again, **do not invoke the function**. Just name it `MAIN` and you're good-to-go!
### Great news!
Great news!
We have finished with *all of the necessary steps* to migrate to v4!
Easy!
While you're here, let's do a couple more things to cleanup your scwrypts (I promise they are also easy and will take less than a few seconds for each)!
### Remove the boilerplate
Were you confused by all that garbage at the top?
Awesome!
Just get rid of any of it you don't use.
While you _probably_ will still need whatever dependencies you already defined, feel free to get rid of empty config lists like `DEPENDENCIES+=()`.
For non-empty lists, the syntax remains the same (use the `+=` and make sure it's an array-type `()` just like before!)
Also you can ditch the `CHECK_ENVIRONMENT`.
While it won't hurt, v4 already does this, so just get rid of it.
Here's my recommended formatting:
```diff
#!/bin/zsh
- #####################################################################
DEPENDENCIES+=(dep-function-a dep-function-b)
- REQUIRED_ENV+=()
use do/awesome/stuff --group my-custom-library
-
- CHECK_ENVIRONMENT
#####################################################################
MAIN() {
echo "do some stuff here"
# ... etc ...
SUCCESS "completed the stuff
}
```
### Get rid of `--help` argument processing
Scwrypts v4 injects the `--help` argument into every zsh scwrypt.
So there's no need to process it manually anymore!
We can now eliminate the help case from any MAIN body or library function:
```diff
MAIN() {
while [[ $# -gt 0 ]]
do
case $1 in
# ... a bunch of cases ...
- -h | --help ) USAGE; return 0 ;;
# ... a bunch of cases ...
esac
shift 1
done
}
```
While you probably weren't doing this, you can also do the same for any logic which splits arguments input like `-abc` which should be read as `-a -b -c`.
If you know how to do this, you know how to get rid of it.
### Write some help docs
Okay this one might take a little bit of time if you haven't done it already (but this is the last recommended step! hang in there and make your stuff better!).
If you _have_ done it already, typically by writing a variable called "USAGE" in your code, maybe consider the _new and improved_ way to write your help strings.
Returning to our original `MAIN()` example, I'll add some options parsing so we should now look something like this:
```sh
#!/bin/zsh
DEPENDENCIES+=(dep-function-a dep-function-b)
use do/awesome/stuff --group my-custom-library
#####################################################################
MAIN() {
local A
local B=false
local ARGS=()
while [[ $# -gt 0 ]]
do
case $1 in
-a | --option-a ) A=$2; shift 1 ;;
-b | --allow-b ) B=true ;;
* ) ARGS+=($1) ;;
esac
shift 1
done
echo "A : $A\nB : $B\nARGS : $ARGS"
}
```
All we have to do is add some usage variables and we're done!
I want to call out a few specific ones:
- `USAGE__options` provides descriptions for CLI flags like `-a` `--some-flag` and `-a <some value>` (reminder, you *don't* need '-h, --help' anymore!)
- `USAGE__args` provides descriptions for non-flag CLI arguments, where order matters (e.g. `cat file-a file-b ... etc`)
- `USAGE__description` provides the human-readable description of what your function does
- `USAGE__usage` you probably don't need to adjust this one, but it will be everything after the `--` in the usage-line. Defaults to include `[...options...]`, but I suppose you might want to write `USAGE__usage+=' [...args...]` if you 1) have args and 2) are really specific about your help strings.
Just add another section to define these values before declaring `MAIN`:
```sh
#!/bin/zsh
DEPENDENCIES+=(dep-function-a dep-function-b)
use do/awesome/stuff --group my-custom-library
#####################################################################
USAGE__options='
-a, --option-a <string> sets the value of the A variable
-b, --allow-b enables the B option
'
# remember there's no specific formatting here, just write it nice
USAGE__args='
N-args All remaining args are inserted into the ARGS variable
'
USAGE__description="
This is my cool example function. It's really neato, but does
very little.
"
#####################################################################
MAIN() {
# ... etc ...
}
```
Now, when we run `scwrypts my sample -- --help`, we get:
```txt
usage: scwrypts my sample -- [...options...]
args:
N-args All remaining args are inserted into the ARGS variable
options:
-a, --option-a <string> sets the value of the A variable
-b, --allow-b enables the B option
-h, --help display this message and exit
This is my cool example function. It's really neato, but does
very little.
```
### All done
No more recommendations at this time.
Someday I'll have an auto-formatter and a language server to help with go-to-definition, but that's still for the future.
Thanks for your time and welcome to v4!

7
plugins/ci/check-all-dependencies
View File

@ -1,9 +1,5 @@
#!/bin/zsh #!/bin/zsh
##################################################################### #####################################################################
DEPENDENCIES+=()
REQUIRED_ENV+=()
#####################################################################
MAIN() { MAIN() {
cd "$SCWRYPTS_ROOT__scwrypts/" cd "$SCWRYPTS_ROOT__scwrypts/"
@ -35,6 +31,3 @@ MAIN() {
echo $DEPENDENCIES | sed 's/ /\n/g' echo $DEPENDENCIES | sed 's/ /\n/g'
CHECK_ENVIRONMENT && SUCCESS "all dependencies satisfied" CHECK_ENVIRONMENT && SUCCESS "all dependencies satisfied"
} }
#####################################################################
MAIN $@

9
plugins/kubectl/get-context
View File

@ -1,16 +1,7 @@
#!/bin/zsh #!/bin/zsh
#####################################################################
DEPENDENCIES+=()
REQUIRED_ENV+=()
use kubectl --group kubectl use kubectl --group kubectl
CHECK_ENVIRONMENT
##################################################################### #####################################################################
MAIN() { MAIN() {
KUBECTL__GET_CONTEXT KUBECTL__GET_CONTEXT
} }
#####################################################################
MAIN $@

9
plugins/kubectl/get-namespace
View File

@ -1,16 +1,7 @@
#!/bin/zsh #!/bin/zsh
#####################################################################
DEPENDENCIES+=()
REQUIRED_ENV+=()
use kubectl --group kubectl use kubectl --group kubectl
CHECK_ENVIRONMENT
##################################################################### #####################################################################
MAIN() { MAIN() {
KUBECTL__GET_NAMESPACE KUBECTL__GET_NAMESPACE
} }
#####################################################################
MAIN $@

10
plugins/kubectl/meta/get-static-redis-definition
View File

@ -1,11 +1,7 @@
#!/bin/zsh #!/bin/zsh
#####################################################################
DEPENDENCIES+=()
REQUIRED_ENV+=()
use redis --group kubectl use redis --group kubectl
CHECK_ENVIRONMENT
##################################################################### #####################################################################
echo $(REDIS --get-static-definition) MAIN() {
echo $(REDIS --get-static-definition)
}

9
plugins/kubectl/serve
View File

@ -1,11 +1,5 @@
#!/bin/zsh #!/bin/zsh
#####################################################################
DEPENDENCIES+=()
REQUIRED_ENV+=()
use kubectl --group kubectl use kubectl --group kubectl
CHECK_ENVIRONMENT
##################################################################### #####################################################################
MAIN() { MAIN() {
@ -53,6 +47,3 @@ MAIN() {
KUBECTL__SERVE KUBECTL__SERVE
} }
#####################################################################
MAIN $@

9
plugins/kubectl/set-context
View File

@ -1,11 +1,5 @@
#!/bin/zsh #!/bin/zsh
#####################################################################
DEPENDENCIES+=()
REQUIRED_ENV+=()
use kubectl --group kubectl use kubectl --group kubectl
CHECK_ENVIRONMENT
##################################################################### #####################################################################
MAIN() { MAIN() {
@ -45,6 +39,3 @@ MAIN() {
KUBECTL__SET_CONTEXT $CONTEXT KUBECTL__SET_CONTEXT $CONTEXT
} }
#####################################################################
MAIN $@

9
plugins/kubectl/set-namespace
View File

@ -1,11 +1,5 @@
#!/bin/zsh #!/bin/zsh
#####################################################################
DEPENDENCIES+=()
REQUIRED_ENV+=()
use kubectl --group kubectl use kubectl --group kubectl
CHECK_ENVIRONMENT
##################################################################### #####################################################################
MAIN() { MAIN() {
@ -45,6 +39,3 @@ MAIN() {
KUBECTL__SET_NAMESPACE $NAMESPACE KUBECTL__SET_NAMESPACE $NAMESPACE
} }
#####################################################################
MAIN $@

20
py/data/convert/csv-to-json.py
View File

@ -1,13 +1,11 @@
#!/usr/bin/env python #!/usr/bin/env python
from py.lib.data.converter import convert from scwrypts import execute
from py.lib.scwrypts import execute
from py.lib.scwrypts.exceptions import ImportedExecutableError
if __name__ != '__main__':
raise ImportedExecutableError()
##################################################################### #####################################################################
from scwrypts.data import convert
description = 'convert csv into json'
parse_args = []
def main(_args, stream): def main(_args, stream):
return convert( return convert(
@ -18,7 +16,5 @@ def main(_args, stream):
) )
##################################################################### #####################################################################
execute(main, if __name__ == '__main__':
description = 'convert csv into json', execute(main, description, parse_args)
parse_args = [],
)

20
py/data/convert/csv-to-yaml.py
View File

@ -1,13 +1,11 @@
#!/usr/bin/env python #!/usr/bin/env python
from py.lib.data.converter import convert from scwrypts import execute
from py.lib.scwrypts import execute
from py.lib.scwrypts.exceptions import ImportedExecutableError
if __name__ != '__main__':
raise ImportedExecutableError()
##################################################################### #####################################################################
from scwrypts.data import convert
description = 'convert csv into yaml'
parse_args = []
def main(_args, stream): def main(_args, stream):
return convert( return convert(
@ -18,7 +16,5 @@ def main(_args, stream):
) )
##################################################################### #####################################################################
execute(main, if __name__ == '__main__':
description = 'convert csv into yaml', execute(main, description, parse_args)
parse_args = [],
)

20
py/data/convert/json-to-csv.py
View File

@ -1,13 +1,11 @@
#!/usr/bin/env python #!/usr/bin/env python
from py.lib.data.converter import convert from scwrypts import execute
from py.lib.scwrypts import execute
from py.lib.scwrypts.exceptions import ImportedExecutableError
if __name__ != '__main__':
raise ImportedExecutableError()
##################################################################### #####################################################################
from scwrypts.data import convert
description = 'convert json into csv'
parse_args = []
def main(_args, stream): def main(_args, stream):
return convert( return convert(
@ -18,7 +16,5 @@ def main(_args, stream):
) )
##################################################################### #####################################################################
execute(main, if __name__ == '__main__':
description = 'convert json into csv', execute(main, description, parse_args)
parse_args = [],
)

20
py/data/convert/json-to-yaml.py
View File

@ -1,13 +1,11 @@
#!/usr/bin/env python #!/usr/bin/env python
from py.lib.data.converter import convert from scwrypts import execute
from py.lib.scwrypts import execute
from py.lib.scwrypts.exceptions import ImportedExecutableError
if __name__ != '__main__':
raise ImportedExecutableError()
##################################################################### #####################################################################
from scwrypts.data import convert
description = 'convert json into yaml'
parse_args = []
def main(_args, stream): def main(_args, stream):
return convert( return convert(
@ -18,7 +16,5 @@ def main(_args, stream):
) )
##################################################################### #####################################################################
execute(main, if __name__ == '__main__':
description = 'convert json into yaml', execute(main, description, parse_args)
parse_args = [],
)

20
py/data/convert/yaml-to-csv.py
View File

@ -1,13 +1,11 @@
#!/usr/bin/env python #!/usr/bin/env python
from py.lib.data.converter import convert from scwrypts import execute
from py.lib.scwrypts import execute
from py.lib.scwrypts.exceptions import ImportedExecutableError
if __name__ != '__main__':
raise ImportedExecutableError()
##################################################################### #####################################################################
from scwrypts.data import convert
description = 'convert yaml into csv'
parse_args = []
def main(_args, stream): def main(_args, stream):
return convert( return convert(
@ -18,7 +16,5 @@ def main(_args, stream):
) )
##################################################################### #####################################################################
execute(main, if __name__ == '__main__':
description = 'convert yaml into csv', execute(main, description, parse_args)
parse_args = [],
)

20
py/data/convert/yaml-to-json.py
View File

@ -1,13 +1,11 @@
#!/usr/bin/env python #!/usr/bin/env python
from py.lib.data.converter import convert from scwrypts import execute
from py.lib.scwrypts import execute
from py.lib.scwrypts.exceptions import ImportedExecutableError
if __name__ != '__main__':
raise ImportedExecutableError()
##################################################################### #####################################################################
from scwrypts.data import convert
description = 'convert yaml into json'
parse_args = []
def main(_args, stream): def main(_args, stream):
return convert( return convert(
@ -18,7 +16,5 @@ def main(_args, stream):
) )
##################################################################### #####################################################################
execute(main, if __name__ == '__main__':
description = 'convert yaml into json', execute(main, description, parse_args)
parse_args = [],
)

101
py/directus/get-items.py
View File

@ -1,16 +1,55 @@
#!/usr/bin/env python #!/usr/bin/env python
from scwrypts import execute
#####################################################################
from json import dumps from json import dumps
from py.lib.fzf import fzf, fzf_tail from scwrypts.fzf import fzf, fzf_tail
from py.lib.http import directus from scwrypts.http import directus
from py.lib.scwrypts import execute
from py.lib.scwrypts.exceptions import ImportedExecutableError
if __name__ != '__main__': description = 'interactive CLI to get data from directus'
raise ImportedExecutableError()
##################################################################### parse_args = [
( ['-c', '--collection'], {
"dest" : 'collection',
"default" : None,
"help" : 'the name of the collection',
"required" : False,
}),
( ['-f', '--filters'], {
"dest" : 'filters',
"default" : None,
"help" : 'as a URL-suffix, filters for the query',
"required" : False,
}),
( ['-d', '--fields'], {
"dest" : 'fields',
"default" : None,
"help" : 'comma-separated list of fields to include',
"required" : False,
}),
( ['-p', '--interactive-prompt'], {
"action" : 'store_true',
"dest" : 'interactive',
"default" : False,
"help" : 'interactively generate filter prompts; implied if no flags are provided',
"required" : False,
}),
( ['--prompt-filters'], {
"action" : 'store_true',
"dest" : 'generate_filters_prompt',
"default" : False,
"help" : '(superceded by -p) only generate filters interactively',
"required" : False,
}),
( ['--prompt-fields'], {
"action" : 'store_true',
"dest" : 'generate_fields_prompt',
"default" : False,
"help" : '(superceded by -p) only generate filters interactively',
"required" : False,
}),
]
def main(args, stream): def main(args, stream):
if {None} == { args.collection, args.filters, args.fields }: if {None} == { args.collection, args.filters, args.fields }:
@ -96,50 +135,6 @@ def _get_or_select_fields(args, collection):
return fields return fields
##################################################################### #####################################################################
execute(main, if __name__ == '__main__':
description = 'interactive CLI to get data from directus', execute(main, description, parse_args)
parse_args = [
( ['-c', '--collection'], {
"dest" : 'collection',
"default" : None,
"help" : 'the name of the collection',
"required" : False,
}),
( ['-f', '--filters'], {
"dest" : 'filters',
"default" : None,
"help" : 'as a URL-suffix, filters for the query',
"required" : False,
}),
( ['-d', '--fields'], {
"dest" : 'fields',
"default" : None,
"help" : 'comma-separated list of fields to include',
"required" : False,
}),
( ['-p', '--interactive-prompt'], {
"action" : 'store_true',
"dest" : 'interactive',
"default" : False,
"help" : 'interactively generate filter prompts; implied if no flags are provided',
"required" : False,
}),
( ['--prompt-filters'], {
"action" : 'store_true',
"dest" : 'generate_filters_prompt',
"default" : False,
"help" : '(superceded by -p) only generate filters interactively',
"required" : False,
}),
( ['--prompt-fields'], {
"action" : 'store_true',
"dest" : 'generate_fields_prompt',
"default" : False,
"help" : '(superceded by -p) only generate filters interactively',
"required" : False,
}),
]
)

70
py/discord/post-message.py
View File

@ -1,16 +1,40 @@
#!/usr/bin/env python #!/usr/bin/env python
from scwrypts import execute
#####################################################################
from json import dumps from json import dumps
from sys import stderr from sys import stderr
from py.lib.http import discord from scwrypts.http import discord
from py.lib.scwrypts import execute
from py.lib.scwrypts.exceptions import ImportedExecutableError
if __name__ != '__main__': description = 'post a message to the indicated discord channel'
raise ImportedExecutableError() parse_args = [
( ['-b', '--body'], {
##################################################################### 'dest' : 'content',
'help' : 'message body',
'required' : False,
}),
( ['-c', '--channel-id'], {
'dest' : 'channel_id',
'help' : 'override default target channel id',
'required' : False,
}),
( ['-w', '--webhook'], {
'dest' : 'webhook',
'help' : 'override default target webhook (takes precedence over -c)',
'required' : False,
}),
( ['--avatar-url'], {
'dest' : 'avatar_url',
'help' : 'override default avatar_url',
'required' : False,
}),
( ['--username'], {
'dest' : 'username',
'help' : 'override default username',
'required' : False,
}),
]
def main(args, stream): def main(args, stream):
if args.content is None: if args.content is None:
@ -29,33 +53,5 @@ def main(args, stream):
##################################################################### #####################################################################
execute(main, if __name__ == '__main__':
description = 'post a message to the indicated discord channel', execute(main, description, parse_args)
parse_args = [
( ['-b', '--body'], {
'dest' : 'content',
'help' : 'message body',
'required' : False,
}),
( ['-c', '--channel-id'], {
'dest' : 'channel_id',
'help' : 'override default target channel id',
'required' : False,
}),
( ['-w', '--webhook'], {
'dest' : 'webhook',
'help' : 'override default target webhook (takes precedence over -c)',
'required' : False,
}),
( ['--avatar-url'], {
'dest' : 'avatar_url',
'help' : 'override default avatar_url',
'required' : False,
}),
( ['--username'], {
'dest' : 'username',
'help' : 'override default username',
'required' : False,
}),
]
)

30
py/hello-world.py
View File

@ -1,27 +1,21 @@
#!/usr/bin/env python #!/usr/bin/env python
from py.lib.scwrypts import execute from scwrypts import execute
from py.lib.scwrypts.exceptions import ImportedExecutableError
if __name__ != '__main__':
raise ImportedExecutableError()
##################################################################### #####################################################################
description = 'a simple "Hello, World!" program'
parse_args = [
( ['-m', '--message'], {
'dest' : 'message',
'default' : 'HELLO WORLD',
'help' : 'message to print',
'required' : False,
}),
]
def main(args, stream): def main(args, stream):
stream.writeline(args.message) stream.writeline(args.message)
##################################################################### #####################################################################
execute(main, if __name__ == '__main__':
description = 'a simple "Hello, World!" program', execute(main, description, parse_args)
parse_args = [
( ['-m', '--message'], {
'dest' : 'message',
'default' : 'HELLO WORLD',
'help' : 'message to print',
'required' : False,
}),
],
)

1
py/lib/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
dist/

3
py/lib/README.md Normal file
View File

@ -0,0 +1,3 @@
# Python Scwrypts
[![Generic Badge](https://img.shields.io/badge/python->=3.9-informational.svg)](https://python.org)
<br>

6
py/lib/__init__.py
View File

@ -1,6 +0,0 @@
import py.lib.data
import py.lib.fzf
import py.lib.http
import py.lib.redis
import py.lib.scwrypts
import py.lib.twilio

1
py/lib/data/__init__.py
View File

@ -1 +0,0 @@
import py.lib.data.converter

1
py/lib/fzf/__init__.py
View File

@ -1 +0,0 @@
from py.lib.fzf.client import fzf, fzf_tail, fzf_head

5
py/lib/http/__init__.py
View File

@ -1,5 +0,0 @@
from py.lib.http.client import get_request_client
import py.lib.http.directus
import py.lib.http.discord
import py.lib.http.linear

20
py/lib/http/client.py
View File

@ -1,20 +0,0 @@
from requests import request
def get_request_client(base_url, headers=None):
if headers is None:
headers = {}
return lambda method, endpoint, **kwargs: request(
method = method,
url = f'{base_url}/{endpoint}',
headers = {
**headers,
**kwargs.get('headers', {}),
},
**{
key: value
for key, value in kwargs.items()
if key != 'headers'
},
)

2
py/lib/http/directus/__init__.py
View File

@ -1,2 +0,0 @@
from py.lib.http.directus.client import *
from py.lib.http.directus.constant import *

56
py/lib/http/directus/client.py
View File

@ -1,56 +0,0 @@
from py.lib.http import get_request_client
from py.lib.scwrypts import getenv
REQUEST = None
COLLECTIONS = None
FIELDS = {}
def request(method, endpoint, **kwargs):
global REQUEST # pylint: disable=global-statement
if REQUEST is None:
REQUEST = get_request_client(
base_url = getenv("DIRECTUS__BASE_URL"),
headers = {
'Authorization': f'bearer {getenv("DIRECTUS__API_TOKEN")}',
}
)
return REQUEST(method, endpoint, **kwargs)
def graphql(query, system=False):
return request(
'POST',
'graphql' if system is True else 'graphql/system',
json={'query': query},
)
def get_collections():
global COLLECTIONS # pylint: disable=global-statement
if COLLECTIONS is None:
COLLECTIONS = [
item['collection']
for item in request(
'GET',
'collections?limit=-1&fields[]=collection',
).json()['data']
]
return COLLECTIONS
def get_fields(collection):
if FIELDS.get(collection) is None:
FIELDS[collection] = [
item['field']
for item in request(
'GET',
f'fields/{collection}?limit=-1&fields[]=field',
).json()['data']
]
return FIELDS[collection]

2
py/lib/http/discord/__init__.py
View File

@ -1,2 +0,0 @@
from py.lib.http.discord.client import *
from py.lib.http.discord.send_message import *

20
py/lib/http/discord/client.py
View File

@ -1,20 +0,0 @@
from py.lib.http import get_request_client
from py.lib.scwrypts import getenv
REQUEST = None
def request(method, endpoint, **kwargs):
global REQUEST # pylint: disable=global-statement
if REQUEST is None:
headers = {}
if (token := getenv("DISCORD__BOT_TOKEN", required = False)) is not None:
headers['Authorization'] = f'Bot {token}'
REQUEST = get_request_client(
base_url = 'https://discord.com/api',
headers = headers,
)
return REQUEST(method, endpoint, **kwargs)

1
py/lib/http/linear/__init__.py
View File

@ -1 +0,0 @@
from py.lib.http.linear.client import *

20
py/lib/http/linear/client.py
View File

@ -1,20 +0,0 @@
from py.lib.http import get_request_client
from py.lib.scwrypts import getenv
REQUEST = None
def request(method, endpoint, **kwargs):
global REQUEST # pylint: disable=global-statement
if REQUEST is None:
REQUEST = get_request_client(
base_url = 'https://api.linear.app',
headers = {
'Authorization': f'bearer {getenv("LINEAR__API_TOKEN")}',
}
)
return REQUEST(method, endpoint, **kwargs)
def graphql(query):
return request('POST', 'graphql', json={'query': query})

60
py/lib/pyproject.toml Normal file
View File

@ -0,0 +1,60 @@
[project]
name = 'scwrypts'
description = 'scwrypts library and invoker'
license = 'GPL-3.0-or-later'
readme = 'README.md'
requires-python = '>=3.10'
authors = [
{ name='yage', email='yage@yage.io' },
]
classifiers = [
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
'Programming Language :: Python :: 3.12',
'License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)',
]
dynamic = ['version']
dependencies = [
'bpython',
'pyfzf',
'pyyaml',
'redis',
'twilio',
]
[project.optional-dependencies]
dev = [
'pylint',
]
test = [
'pytest',
]
[project.urls]
homepage = 'https://github.com/wrynegade/scwrypts'
issues = 'https://github.com/wrynegade/scwrypts/issues'
[build-system]
requires = [
'hatchling',
'versioningit',
]
build-backend = 'hatchling.build'
[tool.hatch.version]
source = 'versioningit'
[tool.hatch.build.targets.wheel]
packages = ['./']
[tool.versioningit]
match = ['v*']

1
py/lib/redis/__init__.py
View File

@ -1 +0,0 @@
from py.lib.redis.client import get_client

17
py/lib/scwrypts/__init__.py
View File

@ -1,6 +1,13 @@
from py.lib.scwrypts.execute import execute '''
from py.lib.scwrypts.getenv import getenv scwrypts
from py.lib.scwrypts.interactive import interactive
from py.lib.scwrypts.run import run
import py.lib.scwrypts.io python library functions and invoker for scwrypts
'''
__all__ = [
'scwrypts',
'execute',
'interactive',
]
from .scwrypts import scwrypts, execute, interactive

1
py/lib/scwrypts/data/__init__.py Normal file
View File

@ -0,0 +1 @@
from .converter import convert

3
py/lib/data/converter.py → py/lib/scwrypts/data/converter.py
View File

@ -4,9 +4,6 @@ import yaml
def convert(input_stream, input_type, output_stream, output_type): def convert(input_stream, input_type, output_stream, output_type):
if input_type == output_type:
raise ValueError('input type and output type are the same')
data = convert_input(input_stream, input_type) data = convert_input(input_stream, input_type)
write_output(output_stream, output_type, data) write_output(output_stream, output_type, data)

73
py/lib/scwrypts/data/test_converter.py Normal file
View File

@ -0,0 +1,73 @@
from io import StringIO
from pytest import raises
from scwrypts.test import generate
from .converter import convert
GENERATE_OPTIONS = {
'depth': 1,
'minimum': -999999,
'maximum': 999999,
'dict_key_types': {str, int},
'csv_columns_minimum': 10,
'csv_columns_maximum': 64,
'csv_rows_minimum': 10,
'csv_rows_maximum': 64,
}
INPUT_TYPES = {'csv', 'json', 'yaml'}
OUTPUT_TYPES = {'csv', 'json', 'yaml'}
def test_convert_to_csv():
for input_type in INPUT_TYPES:
input_stream = generate(input_type, {
**GENERATE_OPTIONS,
'data_types': {bool,int,float,str},
})
if isinstance(input_stream, str):
input_stream = StringIO(input_stream)
convert(input_stream, input_type, StringIO(), 'csv')
def test_convert_to_json():
for input_type in INPUT_TYPES:
input_stream = generate(input_type, GENERATE_OPTIONS)
if isinstance(input_stream, str):
input_stream = StringIO(input_stream)
convert(input_stream, input_type, StringIO(), 'json')
def test_convert_to_yaml():
for input_type in INPUT_TYPES:
input_stream = generate(input_type, GENERATE_OPTIONS)
if isinstance(input_stream, str):
input_stream = StringIO(input_stream)
convert(input_stream, input_type, StringIO(), 'yaml')
def test_convert_deep_json_to_yaml():
input_stream = generate('json', {**GENERATE_OPTIONS, 'depth': 4})
convert(input_stream, 'json', StringIO(), 'yaml')
def test_convert_deep_yaml_to_json():
input_stream = generate('yaml', {**GENERATE_OPTIONS, 'depth': 4})
convert(input_stream, 'yaml', StringIO(), 'json')
def test_convert_output_unsupported():
for input_type in list(INPUT_TYPES):
with raises(ValueError):
convert(StringIO(), input_type, StringIO(), generate(str))
def test_convert_input_unsupported():
for output_type in list(OUTPUT_TYPES):
with raises(ValueError):
convert(StringIO(), generate(str), StringIO(), output_type)

2
py/lib/scwrypts/getenv.py → py/lib/scwrypts/env.py
View File

@ -1,6 +1,6 @@
from os import getenv as os_getenv from os import getenv as os_getenv
from py.lib.scwrypts.exceptions import MissingVariableError from .scwrypts.exceptions import MissingVariableError
def getenv(name, required=True): def getenv(name, required=True):

16
py/lib/scwrypts/exceptions.py
View File

@ -1,16 +0,0 @@
from argparse import ArgumentError
class MissingVariableError(EnvironmentError):
def init(self, name):
super().__init__(f'Missing required environment variable "{name}"')
class ImportedExecutableError(ImportError):
def __init__(self):
super().__init__('executable only; must run through scwrypts')
class MissingFlagAndEnvironmentVariableError(EnvironmentError, ArgumentError):
def __init__(self, flags, env_var):
super().__init__(f'must provide at least one of : {{ flags: {flags} OR {env_var} }}')

1
py/lib/scwrypts/fzf/__init__.py Normal file
View File

@ -0,0 +1 @@
from .client import fzf, fzf_tail, fzf_head

0
py/lib/fzf/client.py → py/lib/scwrypts/fzf/client.py
View File

1
py/lib/scwrypts/http/__init__.py Normal file
View File

@ -0,0 +1 @@
from .client import get_request_client

25
py/lib/scwrypts/http/client.py Normal file
View File

@ -0,0 +1,25 @@
from requests import request
CLIENTS = {}
def get_request_client(base_url, headers=None):
if CLIENTS.get(base_url, None) is None:
if headers is None:
headers = {}
CLIENTS[base_url] = lambda method, endpoint, **kwargs: request(
method = method,
url = f'{base_url}/{endpoint}',
headers = {
**headers,
**kwargs.get('headers', {}),
},
**{
key: value
for key, value in kwargs.items()
if key != 'headers'
},
)
return CLIENTS[base_url]

43
py/lib/scwrypts/http/conftest.py Normal file
View File

@ -0,0 +1,43 @@
from types import SimpleNamespace
from pytest import fixture
from scwrypts.test import generate
from scwrypts.test.character_set import uri
options = {
'str_length_minimum': 8,
'str_length_maximum': 128,
'uuid_output_type': str,
}
def get_request_client_sample_data():
return {
'base_url' : generate(str, options | {'character_set': uri}),
'endpoint' : generate(str, options | {'character_set': uri}),
'method' : generate(str, options),
'response' : generate('requests_Response', options | {'depth': 4}),
'payload' : generate(dict, {
**options,
'depth': 1,
'data_types': { str, 'uuid' },
}),
}
@fixture(name='sample')
def fixture_sample():
return SimpleNamespace(
**get_request_client_sample_data(),
headers = generate(dict, {
**options,
'depth': 1,
'data_types': { str, 'uuid' },
}),
payload_headers = generate(dict, {
**options,
'depth': 1,
'data_types': { str, 'uuid' },
}),
)

20
py/lib/http/directus/constant.py → py/lib/scwrypts/http/directus/__init__.py
View File

@ -1,3 +1,23 @@
'''
basic scwrypts.http client for directus
configured by setting DIRECTUS__BASE_URL and DIRECTUS__API_TOKEN in
scwrypts environment
'''
__all__ = [
'request',
'graphql',
'get_collections',
'get_fields',
'FILTER_OPERATORS',
]
from .client import request
from .graphql import graphql
from .collections import get_collections
from .fields import get_fields
FILTER_OPERATORS = { FILTER_OPERATORS = {
'_eq', '_eq',
'_neq', '_neq',

12
py/lib/scwrypts/http/directus/client.py Normal file
View File

@ -0,0 +1,12 @@
from scwrypts.env import getenv
from .. import get_request_client
def request(method, endpoint, **kwargs):
return get_request_client(
base_url = getenv("DIRECTUS__BASE_URL"),
headers = {
'Authorization': f'bearer {getenv("DIRECTUS__API_TOKEN")}',
}
)(method, endpoint, **kwargs)

18
py/lib/scwrypts/http/directus/collections.py Normal file
View File

@ -0,0 +1,18 @@
from .client import request
COLLECTIONS = None
def get_collections():
global COLLECTIONS # pylint: disable=global-statement
if COLLECTIONS is None:
COLLECTIONS = [
item['collection']
for item in request(
'GET',
'collections?limit=-1&fields[]=collection',
).json()['data']
]
return COLLECTIONS

16
py/lib/scwrypts/http/directus/conftest.py Normal file
View File

@ -0,0 +1,16 @@
from types import SimpleNamespace
from pytest import fixture
from scwrypts.test import generate
from scwrypts.test.character_set import uri
from ..conftest import options, get_request_client_sample_data
@fixture(name='sample')
def fixture_sample():
return SimpleNamespace(
**get_request_client_sample_data(),
api_token = generate(str, options | {'character_set': uri}),
query = generate(str, options),
)

16
py/lib/scwrypts/http/directus/fields.py Normal file
View File

@ -0,0 +1,16 @@
from .client import request
FIELDS = {}
def get_fields(collection):
if FIELDS.get(collection) is None:
FIELDS[collection] = [
item['field']
for item in request(
'GET',
f'fields/{collection}?limit=-1&fields[]=field',
).json()['data']
]
return FIELDS[collection]

9
py/lib/scwrypts/http/directus/graphql.py Normal file
View File

@ -0,0 +1,9 @@
from .client import request
def graphql(query, system=False):
return request(
'POST',
'graphql' if system is False else 'graphql/system',
json={'query': query},
)

43
py/lib/scwrypts/http/directus/test_client.py Normal file
View File

@ -0,0 +1,43 @@
from unittest.mock import patch
from pytest import fixture
from .client import request
def test_directus_request(sample, _response):
assert _response == sample.response
def test_directus_request_client_setup(sample, _response, mock_get_request_client):
mock_get_request_client.assert_called_once_with(
base_url = sample.base_url,
headers = { 'Authorization': f'bearer {sample.api_token}' },
)
#####################################################################
@fixture(name='_response')
def fixture_response(sample):
return request(
method = sample.method,
endpoint = sample.endpoint,
**sample.payload,
)
#####################################################################
@fixture(name='mock_getenv', autouse=True)
def fixture_mock_getenv(sample):
with patch('scwrypts.http.directus.client.getenv',) as mock:
mock.side_effect = lambda name: {
'DIRECTUS__BASE_URL': sample.base_url,
'DIRECTUS__API_TOKEN': sample.api_token,
}[name]
yield mock
@fixture(name='mock_get_request_client', autouse=True)
def fixture_mock_get_request_client(sample):
with patch('scwrypts.http.directus.client.get_request_client') as mock:
mock.return_value = lambda method, endpoint, **kwargs: sample.response
yield mock

45
py/lib/scwrypts/http/directus/test_graphql.py Normal file
View File

@ -0,0 +1,45 @@
from unittest.mock import patch
from pytest import fixture
from .graphql import graphql
def test_directus_graphql(sample, _response, _mock_request):
assert _response == sample.response
def test_directus_graphql_request_payload(sample, _response, _mock_request):
_mock_request.assert_called_once_with(
'POST',
'graphql',
json = {'query': sample.query},
)
def test_directus_graphql_system(sample, _response_system):
assert _response_system == sample.response
def test_directus_graphql_system_request_payload(sample, _response_system, _mock_request):
_mock_request.assert_called_once_with(
'POST',
'graphql/system',
json = {'query': sample.query},
)
#####################################################################
@fixture(name='_response')
def fixture_response(sample, _mock_request):
return graphql(sample.query)
@fixture(name='_response_system')
def fixture_response_system(sample, _mock_request):
return graphql(sample.query, system=True)
#####################################################################
@fixture(name='_mock_request')
def fixture_mock_request(sample):
with patch('scwrypts.http.directus.graphql.request') as mock:
mock.return_value = sample.response
yield mock

14
py/lib/scwrypts/http/discord/__init__.py Normal file
View File

@ -0,0 +1,14 @@
'''
basic scwrypts.http client for discord
configured by setting various DISCORD__* options in the
scwrypts environment
'''
__all__ = [
'request',
'send_message',
]
from .client import request
from .send_message import send_message

15
py/lib/scwrypts/http/discord/client.py Normal file
View File

@ -0,0 +1,15 @@
from scwrypts.env import getenv
from .. import get_request_client
def request(method, endpoint, **kwargs):
headers = {}
if (token := getenv("DISCORD__BOT_TOKEN", required = False)) is not None:
headers['Authorization'] = f'Bot {token}'
return get_request_client(
base_url = 'https://discord.com/api',
headers = headers,
)(method, endpoint, **kwargs)

25
py/lib/scwrypts/http/discord/conftest.py Normal file
View File

@ -0,0 +1,25 @@
from string import ascii_letters, digits
from types import SimpleNamespace
from pytest import fixture
from scwrypts.test import generate
from scwrypts.test.character_set import uri
from ..conftest import options, get_request_client_sample_data
@fixture(name='sample')
def fixture_sample():
return SimpleNamespace(
**{
**get_request_client_sample_data(),
'base_url': 'https://discord.com/api',
},
bot_token = generate(str, options | {'character_set': uri}),
username = generate(str, options | {'character_set': ascii_letters + digits}),
avatar_url = generate(str, options | {'character_set': uri}),
webhook = generate(str, options | {'character_set': uri}),
channel_id = generate(str, options | {'character_set': uri}),
content_header = generate(str, options),
content_footer = generate(str, options),
content = generate(str, options),
)

4
py/lib/http/discord/send_message.py → py/lib/scwrypts/http/discord/send_message.py
View File

@ -1,5 +1,5 @@
from py.lib.scwrypts import getenv from scwrypts.env import getenv
from py.lib.http.discord import request from .client import request
def send_message(content, channel_id=None, webhook=None, username=None, avatar_url=None, **kwargs): def send_message(content, channel_id=None, webhook=None, username=None, avatar_url=None, **kwargs):
if username is None: if username is None:

54
py/lib/scwrypts/http/discord/test_client.py Normal file
View File

@ -0,0 +1,54 @@
from unittest.mock import patch
from pytest import fixture
from .client import request
def test_discord_request(sample, _response):
assert _response == sample.response
def test_discord_request_client_setup(sample, mock_get_request_client, _mock_getenv, _response):
mock_get_request_client.assert_called_once_with(
base_url = sample.base_url,
headers = { 'Authorization': f'Bot {sample.bot_token}' },
)
def test_discord_request_client_setup_public(sample, mock_get_request_client, _mock_getenv_optional, _response):
mock_get_request_client.assert_called_once_with(
base_url = sample.base_url,
headers = {},
)
#####################################################################
@fixture(name='_response')
def fixture_response(sample):
return request(
method = sample.method,
endpoint = sample.endpoint,
**sample.payload,
)
#####################################################################
@fixture(name='mock_get_request_client', autouse=True)
def fixture_mock_get_request_client(sample):
with patch('scwrypts.http.discord.client.get_request_client') as mock:
mock.return_value = lambda method, endpoint, **kwargs: sample.response
yield mock
@fixture(name='_mock_getenv')
def fixture_mock_getenv(sample):
with patch('scwrypts.http.discord.client.getenv',) as mock:
mock.side_effect = lambda name, **kwargs: {
'DISCORD__BOT_TOKEN': sample.bot_token,
}[name]
yield mock
@fixture(name='_mock_getenv_optional')
def fixture_mock_getenv_optional():
with patch('scwrypts.http.discord.client.getenv',) as mock:
mock.side_effect = lambda name, **kwargs: None
yield mock

91
py/lib/scwrypts/http/discord/test_send_message.py Normal file
View File

@ -0,0 +1,91 @@
from unittest.mock import patch
from pytest import fixture, raises
from .send_message import send_message
def test_discord_send_message(sample, mock_request, _mock_getenv):
expected = get_default_called_with(sample)
assert send_message(sample.content) == sample.response
mock_request.assert_called_once_with(**expected)
def test_discord_send_message_without_username(sample, mock_request, _mock_getenv):
sample.username = None
expected = get_default_called_with(sample)
del expected['json']['username']
assert send_message(sample.content) == sample.response
mock_request.assert_called_once_with(**expected)
def test_discord_send_message_without_avatar_url(sample, mock_request, _mock_getenv):
sample.avatar_url = None
expected = get_default_called_with(sample)
del expected['json']['avatar_url']
assert send_message(sample.content) == sample.response
mock_request.assert_called_once_with(**expected)
def test_discord_send_message_to_channel_id(sample, mock_request, _mock_getenv):
sample.webhook = None
expected = get_default_called_with(sample)
expected['endpoint'] = f'channels/{sample.channel_id}/messages'
assert send_message(sample.content) == sample.response
mock_request.assert_called_once_with(**expected)
def test_discord_send_message_without_content_header(sample, mock_request, _mock_getenv):
sample.content_header = None
expected = get_default_called_with(sample)
expected['json']['content'] = f'{sample.content}{sample.content_footer}'
assert send_message(sample.content) == sample.response
mock_request.assert_called_once_with(**expected)
def test_discord_send_message_without_content_footer(sample, mock_request, _mock_getenv):
sample.content_footer = None
expected = get_default_called_with(sample)
expected['json']['content'] = f'{sample.content_header}{sample.content}'
assert send_message(sample.content) == sample.response
mock_request.assert_called_once_with(**expected)
def test_discord_send_message_error(sample, mock_request, _mock_getenv):
with raises(ValueError):
sample.webhook = None
sample.channel_id = None
send_message(sample.content)
#####################################################################
def get_default_called_with(sample):
return {
'method': 'POST',
'endpoint': f'webhooks/{sample.webhook}',
'json': {
'content': f'{sample.content_header}{sample.content}{sample.content_footer}',
'username': sample.username,
'avatar_url': sample.avatar_url,
},
}
@fixture(name='mock_request', autouse=True)
def fixture_mock_request(sample):
with patch('scwrypts.http.discord.send_message.request') as mock:
mock.return_value = sample.response
yield mock
@fixture(name='_mock_getenv')
def fixture_mock_getenv(sample):
with patch('scwrypts.http.discord.send_message.getenv',) as mock:
mock.side_effect = lambda name, **kwargs: {
'DISCORD__DEFAULT_USERNAME': sample.username,
'DISCORD__DEFAULT_AVATAR_URL': sample.avatar_url,
'DISCORD__DEFAULT_WEBHOOK': sample.webhook,
'DISCORD__DEFAULT_CHANNEL_ID': sample.channel_id,
'DISCORD__CONTENT_HEADER': sample.content_header,
'DISCORD__CONTENT_FOOTER': sample.content_footer,
}[name]
yield mock

14
py/lib/scwrypts/http/linear/__init__.py Normal file
View File

@ -0,0 +1,14 @@
'''
basic scwrypts.http client for linear
configured by setting the LINEAR__API_TOKEN option in the
scwrypts environment
'''
__all__ = [
'request',
'graphql',
]
from .client import request
from .graphql import graphql

12
py/lib/scwrypts/http/linear/client.py Normal file
View File

@ -0,0 +1,12 @@
from scwrypts.env import getenv
from .. import get_request_client
def request(method, endpoint, **kwargs):
return get_request_client(
base_url = 'https://api.linear.app',
headers = {
'Authorization': f'bearer {getenv("LINEAR__API_TOKEN")}',
},
)(method, endpoint, **kwargs)

19
py/lib/scwrypts/http/linear/conftest.py Normal file
View File

@ -0,0 +1,19 @@
from string import ascii_letters, digits
from types import SimpleNamespace
from pytest import fixture
from scwrypts.test import generate
from scwrypts.test.character_set import uri
from ..conftest import options, get_request_client_sample_data
@fixture(name='sample')
def fixture_sample():
return SimpleNamespace(
**{
**get_request_client_sample_data(),
'base_url': 'https://api.linear.app',
},
api_token = generate(str, options | {'character_set': uri}),
query = generate(str, options),
)

5
py/lib/scwrypts/http/linear/graphql.py Normal file
View File

@ -0,0 +1,5 @@
from .client import request
def graphql(query):
return request('POST', 'graphql', json={'query': query})

42
py/lib/scwrypts/http/linear/test_client.py Normal file
View File

@ -0,0 +1,42 @@
from unittest.mock import patch
from pytest import fixture
from .client import request
def test_discord_request(sample, _response):
assert _response == sample.response
def test_discord_request_client_setup(sample, mock_get_request_client, _response):
mock_get_request_client.assert_called_once_with(
base_url = sample.base_url,
headers = { 'Authorization': f'bearer {sample.api_token}' },
)
#####################################################################
@fixture(name='_response')
def fixture_response(sample):
return request(
method = sample.method,
endpoint = sample.endpoint,
**sample.payload,
)
#####################################################################
@fixture(name='mock_get_request_client', autouse=True)
def fixture_mock_get_request_client(sample):
with patch('scwrypts.http.linear.client.get_request_client') as mock:
mock.return_value = lambda method, endpoint, **kwargs: sample.response
yield mock
@fixture(name='mock_getenv', autouse=True)
def fixture_mock_getenv(sample):
with patch('scwrypts.http.linear.client.getenv',) as mock:
mock.side_effect = lambda name, **kwargs: {
'LINEAR__API_TOKEN': sample.api_token,
}[name]
yield mock

35
py/lib/scwrypts/http/linear/test_graphql.py Normal file
View File

@ -0,0 +1,35 @@
from unittest.mock import patch
from pytest import fixture
from .graphql import graphql
def test_directus_graphql(sample, _response, _mock_request):
assert _response == sample.response
def test_directus_graphql_request_payload(sample, _response, _mock_request):
_mock_request.assert_called_once_with(
'POST',
'graphql',
json = {'query': sample.query},
)
#####################################################################
@fixture(name='_response')
def fixture_response(sample, _mock_request):
return graphql(sample.query)
@fixture(name='_response_system')
def fixture_response_system(sample, _mock_request):
return graphql(sample.query)
#####################################################################
@fixture(name='_mock_request')
def fixture_mock_request(sample):
with patch('scwrypts.http.linear.graphql.request') as mock:
mock.return_value = sample.response
yield mock

55
py/lib/scwrypts/http/test_client.py Normal file
View File

@ -0,0 +1,55 @@
from unittest.mock import patch
from pytest import fixture
from .client import get_request_client
def test_request_client(sample, _response_basic):
assert _response_basic == sample.response
def test_request_client_forwards_default_headers(sample, mock_request, _response_basic):
mock_request.assert_called_once_with(
method = sample.method,
url = f'{sample.base_url}/{sample.endpoint}',
headers = sample.headers,
)
def test_get_request_client_payload(sample, _response_payload):
assert _response_payload == sample.response
def test_request_client_forwards_payload_headers(sample, mock_request, _response_payload):
assert mock_request.call_args.kwargs['headers'] == sample.headers | sample.payload_headers
#####################################################################
@fixture(name='mock_request', autouse=True)
def fixture_mock_request(sample):
with patch('scwrypts.http.client.request') as mock:
mock.return_value = sample.response
yield mock
@fixture(name='request_client', autouse=True)
def fixture_request_client(sample):
return get_request_client(sample.base_url, sample.headers)
#####################################################################
@fixture(name='_response_basic')
def fixture_response_basic(sample, request_client):
return request_client(
method = sample.method,
endpoint = sample.endpoint,
)
@fixture(name='_response_payload')
def fixture_response_payload(sample, request_client):
return request_client(
method = sample.method,
endpoint = sample.endpoint,
**{
**sample.payload,
'headers': sample.payload_headers,
},
)

1
py/lib/scwrypts/io/__init__.py Normal file
View File

@ -0,0 +1 @@
from .combined_io_stream import get_combined_stream, add_io_arguments

68
py/lib/scwrypts/io.py → py/lib/scwrypts/io/combined_io_stream.py
View File

@ -2,7 +2,47 @@ from contextlib import contextmanager
from pathlib import Path from pathlib import Path
from sys import stdin, stdout, stderr from sys import stdin, stdout, stderr
from py.lib.scwrypts.getenv import getenv from scwrypts.env import getenv
@contextmanager
def get_combined_stream(input_file=None, output_file=None):
'''
context manager to open an "input_file" and "output_file"
But the "files" can be pipe-streams, stdin/stdout, or even
actual files! Helpful when trying to write CLI scwrypts
which would like to accept all kinds of input and output
configurations.
'''
with get_stream(input_file, 'r') as input_stream, get_stream(output_file, 'w+') as output_stream:
yield CombinedStream(input_stream, output_stream)
def add_io_arguments(parser, allow_input=True, allow_output=True):
'''
slap these puppies onto your argparse.ArgumentParser to
allow easy use of the get_combined_stream at the command line
'''
if allow_input:
parser.add_argument(
'-i', '--input-file',
dest = 'input_file',
default = None,
help = 'path to input file; omit for stdin',
required = False,
)
if allow_output:
parser.add_argument(
'-o', '--output-file',
dest = 'output_file',
default = None,
help = 'path to output file; omit for stdout',
required = False,
)
#####################################################################
@contextmanager @contextmanager
@ -34,32 +74,6 @@ def get_stream(filename=None, mode='r', encoding='utf-8', verbose=False, **kwarg
stdout.flush() stdout.flush()
def add_io_arguments(parser, toggle_input=True, toggle_output=True):
if toggle_input:
parser.add_argument(
'-i', '--input-file',
dest = 'input_file',
default = None,
help = 'path to input file; omit for stdin',
required = False,
)
if toggle_output:
parser.add_argument(
'-o', '--output-file',
dest = 'output_file',
default = None,
help = 'path to output file; omit for stdout',
required = False,
)
@contextmanager
def get_combined_stream(input_file=None, output_file=None):
with get_stream(input_file, 'r') as input_stream, get_stream(output_file, 'w+') as output_stream:
yield CombinedStream(input_stream, output_stream)
class CombinedStream: class CombinedStream:
def __init__(self, input_stream, output_stream): def __init__(self, input_stream, output_stream):
self.input = input_stream self.input = input_stream

1
py/lib/scwrypts/redis/__init__.py Normal file
View File

@ -0,0 +1 @@
from .client import get_client

2
py/lib/redis/client.py → py/lib/scwrypts/redis/client.py
View File

@ -1,6 +1,6 @@
from redis import StrictRedis from redis import StrictRedis
from py.lib.scwrypts import getenv from scwrypts.env import getenv
CLIENT = None CLIENT = None

22
py/lib/scwrypts/run.py
View File

@ -1,22 +0,0 @@
from os import getenv
from pathlib import Path
from subprocess import run as subprocess_run
def run(scwrypt_name, *args):
DEPTH = int(getenv('SUBSCWRYPT', '0'))
DEPTH += 1
SCWRYPTS_EXE = Path(__file__).parents[3] / 'scwrypts'
ARGS = ' '.join([str(x) for x in args])
print(f'SUBSCWRYPT={DEPTH} {SCWRYPTS_EXE} {scwrypt_name} -- {ARGS}')
print(f'\n {"--"*DEPTH} ({DEPTH}) BEGIN SUBSCWRYPT : {Path(scwrypt_name).name}')
subprocess_run(
f'SUBSCWRYPT={DEPTH} {SCWRYPTS_EXE} {scwrypt_name} -- {ARGS}',
shell=True,
executable='/bin/zsh',
check=False,
)
print(f' {"--"*DEPTH} ({DEPTH}) END SUBSCWRYPT : {Path(scwrypt_name).name}\n')

23
py/lib/scwrypts/scwrypts/__init__.py Normal file
View File

@ -0,0 +1,23 @@
'''
scwrypts meta-configuration
provides a helpful three ways to run "scwrypts"
'scwrypts' is an agnostic, top-level executor allowing any scwrypt to be called from python workflows
'execute' is the default context set-up for python-based scwrypts
'interactive' is a context set-up for interactive, python-based scwrypts
after execution, you are dropped in a bpython shell with all the variables
configured during main() execution
'''
__all__ = [
'scwrypts',
'execute',
'interactive',
]
from .scwrypts import scwrypts
from .execute import execute
from .interactive import interactive

26
py/lib/scwrypts/scwrypts/exceptions.py Normal file
View File

@ -0,0 +1,26 @@
from argparse import ArgumentError
class MissingVariableError(EnvironmentError):
def init(self, name):
super().__init__(f'Missing required environment variable "{name}"')
class MissingFlagAndEnvironmentVariableError(EnvironmentError, ArgumentError):
def __init__(self, flags, env_var):
super().__init__(f'must provide at least one of : {{ flags: {flags} OR {env_var} }}')
class MissingScwryptsExecutableError(EnvironmentError):
def __init__(self):
super().__init__(f'scwrypts must be installed and available on your PATH')
class BadScwryptsLookupError(ValueError):
def __init__(self):
super().__init__('must provide name/group/type or scwrypt lookup patterns')
class MissingScwryptsGroupOrTypeError(ValueError):
def __init__(self, group, _type):
super().__init__(f'missing required group or type (group={group} | type={_type}')

9
py/lib/scwrypts/execute.py → py/lib/scwrypts/scwrypts/execute.py
View File

@ -1,9 +1,12 @@
from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter
from py.lib.scwrypts.io import get_combined_stream, add_io_arguments from scwrypts.io import get_combined_stream, add_io_arguments
def execute(main, description=None, parse_args=None, toggle_input=True, toggle_output=True): def execute(main, description=None, parse_args=None, allow_input=True, allow_output=True):
'''
API to initiate a python-based scwrypt
'''
if parse_args is None: if parse_args is None:
parse_args = [] parse_args = []
@ -12,7 +15,7 @@ def execute(main, description=None, parse_args=None, toggle_input=True, toggle_o
formatter_class = ArgumentDefaultsHelpFormatter, formatter_class = ArgumentDefaultsHelpFormatter,
) )
add_io_arguments(parser, toggle_input, toggle_output) add_io_arguments(parser, allow_input, allow_output)
for a in parse_args: for a in parse_args:
parser.add_argument(*a[0], **a[1]) parser.add_argument(*a[0], **a[1])

3
py/lib/scwrypts/interactive.py → py/lib/scwrypts/scwrypts/interactive.py
View File

@ -2,6 +2,9 @@ from bpython import embed
def interactive(variable_descriptions): def interactive(variable_descriptions):
'''
main() decorator to drop to interactive python environment upon completion
'''
def outer(function): def outer(function):
def inner(*args, **kwargs): def inner(*args, **kwargs):

48
py/lib/scwrypts/scwrypts/scwrypts.py Normal file
View File

@ -0,0 +1,48 @@
from os import getenv
from shutil import which
from subprocess import run
from .exceptions import MissingScwryptsExecutableError, BadScwryptsLookupError, MissingScwryptsGroupOrTypeError
def scwrypts(*args, patterns=None, name=None, group=None, _type=None, log_level=None):
'''
top-level scwrypts invoker from python
- patterns allows for pattern-based scwrypt lookup
- name/group/type allos for precise-match lookup
*args should be a list of strings and is forwarded to the
invoked scwrypt
see 'scwrypts --help' for more information
'''
executable = which('scwrypts')
if executable is None:
raise MissingScwryptsExecutableError()
if patterns is None and name is None:
raise BadScwryptsLookupError()
pre_args = []
if name is None:
pre_args += patterns
else:
pre_args += ['--name', name, '--group', group, '--type', _type]
if group is None or _type is None:
raise MissingScwryptsGroupOrTypeError(group, _type)
if log_level is not None:
pre_args += ['--log-level', log_level]
depth = getenv('SUBSCWRYPT', '')
if depth != '':
depth = int(depth) + 1
return run(
f'SUBSCWRYPT={depth} {executable} {pre_args} -- {" ".join(args)}',
shell=True,
executable='/bin/zsh',
check=False,
)

10
py/lib/scwrypts/test/__init__.py Normal file
View File

@ -0,0 +1,10 @@
'''
automated testing utilties, but primarily a random data generator
'''
__all__ = [
'generate',
]
from .generate import generate
from .character_set import *

13
py/lib/scwrypts/test/character_set.py Normal file
View File

@ -0,0 +1,13 @@
'''
string constants typically used for randomly generated data
the 'string' standard library already contains many character sets,
but not these :)
'''
__all__ = [
'uri',
]
uri = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~:/?#[]@!$&\'()*+,;='

10
py/lib/scwrypts/test/exceptions.py Normal file
View File

@ -0,0 +1,10 @@
class GeneratorError(Exception):
pass
class NoDataTypeError(GeneratorError, ValueError):
def __init__(self):
super().__init__('must provide at least one data type (either "data_type" or "data_types")')
class BadGeneratorTypeError(GeneratorError, ValueError):
def __init__(self, data_type):
super().__init__(f'no generator exists for data type "{data_type}"')

372
py/lib/scwrypts/test/generate.py Normal file
View File

@ -0,0 +1,372 @@
from csv import writer, QUOTE_NONNUMERIC
from io import StringIO
from json import dumps, loads
from random import randint, uniform, choice
from re import sub
from string import printable
from typing import Hashable, Callable
from uuid import uuid4
from requests import Response, status_codes
from yaml import safe_dump
from .exceptions import NoDataTypeError, BadGeneratorTypeError
DEFAULT_OPTIONS = {
'data_types': None,
'minimum': 0,
'maximum': 64,
'depth': 1,
'character_set': None,
'bool_nullable': False,
'str_length': None,
'str_length_minimum': 0,
'str_length_maximum': 64,
'uuid_output_type': 'uuid', # str or 'uuid'
'list_length': 8,
'set_length': 8,
'dict_length': 8,
'csv_bool_nullable': True,
'csv_columns': None,
'csv_columns_minimum': 1,
'csv_columns_maximum': 16,
'csv_rows': None,
'csv_rows_minimum': 2,
'csv_rows_maximum': 16,
'csv_output_type': 'stringio', # str or 'stringio'
'json_initial_type': dict, # typically dict or list
'json_bool_nullable': True,
'json_output_type': 'stringio', # str or 'stringio'
'yaml_initial_type': dict, # typically dict or list
'yaml_bool_nullable': True,
'yaml_use_default_flow_style': False,
'yaml_output_type': 'stringio', # str or 'stringio'
'requests_response_status_code': status_codes.codes[200],
}
def generate(data_type=None, options=None):
'''
generate random data with the call of a function
use data_type to generate a single value
use options to set generation options (key = type, value = kwargs)
use options.data_types and omit data_type to generate a random type
'''
if options is None:
options = {}
options = DEFAULT_OPTIONS | options
if data_type is None:
if options['data_types'] is None or len(options['data_types']) == 0:
raise NoDataTypeError()
return generate(
data_type=choice(list(options['data_types'])),
options=options,
)
if not isinstance(data_type, str):
data_type = data_type.__name__
if data_type not in Generator.get_supported_data_types():
raise BadGeneratorTypeError(data_type)
return getattr(Generator, f'_{data_type}')(options)
#####################################################################
SUPPORTED_DATA_TYPES = None
class Generator:
@classmethod
def get_supported_data_types(cls):
global SUPPORTED_DATA_TYPES # pylint: disable=global-statement
if SUPPORTED_DATA_TYPES is None:
SUPPORTED_DATA_TYPES = {
sub('^_', '', data_type)
for data_type, method in Generator.__dict__.items()
if isinstance(method, staticmethod)
}
return SUPPORTED_DATA_TYPES
#####################################################################
@classmethod
def filter_data_types(cls, options, filters=None):
'''
returns an options dict with appropriately filtered data_types
if data_types are not yet defined, starts with all supported data_types
'''
if options['data_types'] is None:
options['data_types'] = Generator.get_supported_data_types()
if filters is None or len(filters) == 0:
return options
return {
**options,
'data_types': set(filter(
lambda data_type: all(( f(data_type, options) for f in filters )),
options['data_types'],
)),
}
class Filters:
@staticmethod
def hashable(data_type, _options):
if isinstance(data_type, Callable):
return isinstance(data_type(), Hashable)
if not isinstance(data_type, str):
data_type = data_type.__name__
return data_type in { 'bool', 'int', 'float', 'chr', 'str', 'uuid' }
@staticmethod
def filelike(data_type, _options):
return data_type in { 'csv', 'json', 'yaml' }
@staticmethod
def complex(data_type, _options):
return data_type in { 'requests_Response' }
@staticmethod
def basic(data_type, options):
return all([
not Generator.Filters.filelike(data_type, options),
not Generator.Filters.complex(data_type, options),
])
@staticmethod
def pythonset(data_type, _options):
if not isinstance(data_type, str):
data_type = data_type.__name__
return data_type == 'set'
@staticmethod
def csvsafe(data_type, options):
options['depth'] = max(1, options['depth'])
return all([
Generator.Filters.basic(data_type, options),
not Generator.Filters.pythonset(data_type, options),
])
@staticmethod
def jsonsafe(data_type, options):
return all([
Generator.Filters.basic(data_type, options),
not Generator.Filters.pythonset(data_type, options),
])
@staticmethod
def yamlsafe(data_type, options):
return all([
Generator.Filters.basic(data_type, options),
not Generator.Filters.pythonset(data_type, options),
])
#####################################################################
@classmethod
def get_option_with_range(cls, options, option_key, data_type=int):
'''
typically an integer range, allows both:
- setting a fixed configuration (e.g. 'str_length')
- allowing a configuration range (e.g. 'str_length_minimum' and 'str_length_maximum')
'''
fixed = options.get(option_key, None)
if fixed is not None:
return fixed
return generate(data_type, {
'minimum': options[f'{option_key}_minimum'],
'maximum': options[f'{option_key}_maximum'],
})
#####################################################################
@staticmethod
def _bool(options):
return choice([True, False, None]) if options['bool_nullable'] else choice([True, False])
@staticmethod
def _int(options):
return randint(options['minimum'], options['maximum'])
@staticmethod
def _float(options):
return uniform(options['minimum'], options['maximum'])
@staticmethod
def _chr(options):
character_set = options['character_set']
return choice(character_set) if character_set is not None else chr(randint(0,65536))
@staticmethod
def _str(options):
return ''.join((
generate(chr, options)
for _ in range(Generator.get_option_with_range(options, 'str_length'))
))
@staticmethod
def _uuid(options):
'''
creates a UUID object or a str containing a uuid (v4)
'''
uuid = uuid4()
return str(uuid) if options['uuid_output_type'] == str else uuid
@staticmethod
def _list(options):
if options['depth'] <= 0:
return []
options['depth'] -= 1
options = Generator.filter_data_types(options, [
Generator.Filters.basic,
])
return [ generate(None, {**options}) for _ in range(options['list_length']) ]
@staticmethod
def _set(options):
if options['depth'] <= 0:
return set()
options['depth'] -= 1
options = Generator.filter_data_types(options, [
Generator.Filters.hashable,
])
return { generate(None, options) for _ in range(options['set_length']) }
@staticmethod
def _dict(options):
if options['depth'] <= 0:
return {}
options['depth'] -= 1
options = Generator.filter_data_types(options, [
Generator.Filters.basic,
])
key_options = Generator.filter_data_types(options, [
Generator.Filters.hashable,
])
if len(options['data_types']) == 0 or len(key_options['data_types']) == 0:
return {}
return {
generate(None, key_options): generate(None, options)
for _ in range(options['dict_length'])
}
@staticmethod
def _csv(options):
'''
creates a StringIO object containing csv data
'''
if options['character_set'] is None:
options['character_set'] = printable
options['bool_nullable'] = options['csv_bool_nullable']
options = Generator.filter_data_types(options, [
Generator.Filters.csvsafe,
])
columns = Generator.get_option_with_range(options, 'csv_columns')
rows = Generator.get_option_with_range(options, 'csv_rows')
csv = StringIO()
csv_writer = writer(csv, quoting=QUOTE_NONNUMERIC)
options['list_length'] = columns
[ # pylint: disable=expression-not-assigned
csv_writer.writerow(generate(list, options))
for _ in range(rows)
]
csv.seek(0)
return csv.getvalue() if options['csv_output_type'] == str else csv
@staticmethod
def _json(options):
'''
creates a StringIO object or str containing json data
'''
if options['character_set'] is None:
options['character_set'] = printable
options['bool_nullable'] = options['json_bool_nullable']
options['uuid_output_type'] = str
options = Generator.filter_data_types(options, [
Generator.Filters.jsonsafe,
])
json = dumps(generate(
options['json_initial_type'],
{**options},
))
return json if options['json_output_type'] == str else StringIO(json)
@staticmethod
def _yaml(options):
'''
creates a StringIO object or str containing yaml data
'''
if options['character_set'] is None:
options['character_set'] = printable
options['bool_nullable'] = options['yaml_bool_nullable']
options['uuid_output_type'] = str
options = Generator.filter_data_types(options, [
Generator.Filters.yamlsafe,
])
yaml = StringIO()
safe_dump(
generate(options['yaml_initial_type'], {**options}),
yaml,
default_flow_style=options['yaml_use_default_flow_style'],
)
yaml.seek(0)
return yaml.getvalue() if options['yaml_output_type'] == str else yaml
@staticmethod
def _requests_Response(options):
'''
creates a requests.Response-like object containing json data
'''
options['json_output_type'] = str
response = Response()
response.status_code = options['requests_response_status_code']
json = loads(generate('json', options))
response.json = lambda: json
return response

54
py/lib/scwrypts/test/test_generate.py Normal file
View File

@ -0,0 +1,54 @@
from os import getenv
from pprint import pprint
from random import randint
from .generate import generate, Generator
ITERATIONS = int(
getenv(
'PYTEST_ITERATIONS__scwrypts__test__generator',
getenv('PYTEST_ITERATIONS', '99'), # CI should use at least 999
)
)
FILE_LIKE_DATA_TYPES = { 'csv', 'json', 'yaml' }
def test_generate(): # generators should be quick and "just work" (no Exceptions)
print()
for data_type in Generator.get_supported_data_types():
print(f'------- {data_type} -------')
sample = generate(data_type)
pprint(sample.getvalue() if data_type in {'csv', 'json', 'yaml'} else sample)
for _ in range(ITERATIONS):
generate(data_type)
def test_generate_depth_deep():
for data_type in Generator.get_supported_data_types():
generate(data_type, {'depth': 4})
def test_generate_depth_shallow():
for data_type in Generator.get_supported_data_types():
generate(data_type, {'depth': randint(-999, 0)})
def test_generate_range_all():
for data_type in Generator.get_supported_data_types():
generate(data_type, {'minimum': -99, 'maximum': 99})
def test_generate_range_positive():
for data_type in Generator.get_supported_data_types():
generate(data_type, {'minimum': 1, 'maximum': 99})
def test_generate_range_zero():
for data_type in Generator.get_supported_data_types():
generate(data_type, {'minimum': 3, 'maximum': 3})
def test_generate_range_negative():
for data_type in Generator.get_supported_data_types():
generate(data_type, {'minimum': -99, 'maximum': -1})
def test_generate_bool_nullable():
for data_type in Generator.get_supported_data_types():
generate(data_type, {'bool_nullable': True})

13
py/lib/scwrypts/twilio/__init__.py Normal file
View File

@ -0,0 +1,13 @@
'''
loads the twilio.rest.Client by referencing TWILIO__API_KEY,
TWILIO__API_SECRET, and TWILIO__ACCOUNT_SID in your scwrypts
environment
'''
__all__ = [
'get_client',
'send_sms',
]
from .client import get_client
from .send_sms import send_sms

2
py/lib/twilio/client.py → py/lib/scwrypts/twilio/client.py
View File

@ -1,6 +1,6 @@
from twilio.rest import Client from twilio.rest import Client
from py.lib.scwrypts import getenv from scwrypts.env import getenv
CLIENT = None CLIENT = None

2
py/lib/twilio/send_sms.py → py/lib/scwrypts/twilio/send_sms.py
View File

@ -1,7 +1,7 @@
from json import dumps from json import dumps
from time import sleep from time import sleep
from py.lib.twilio.client import get_client from .client import get_client
def send_sms(to, from_, body, max_char_count=300, stream=None): def send_sms(to, from_, body, max_char_count=300, stream=None):

2
py/lib/twilio/__init__.py
View File

@ -1,2 +0,0 @@
from py.lib.twilio.client import get_client
from py.lib.twilio.send_sms import send_sms

43
py/linear/comment.py
View File

@ -1,13 +1,22 @@
#!/usr/bin/env python #!/usr/bin/env python
from py.lib.http.linear import graphql from scwrypts import execute
from py.lib.scwrypts import execute
from py.lib.scwrypts.exceptions import ImportedExecutableError
if __name__ != '__main__':
raise ImportedExecutableError()
##################################################################### #####################################################################
from scwrypts.http.linear import graphql
description = 'comment on an issue in linear.app'
parse_args = [
( ['-d', '--issue-id'], {
'dest' : 'issue_id',
'help' : 'issue short-code (e.g. CLOUD-319)',
'required' : True,
}),
( ['-m', '--message'], {
'dest' : 'message',
'help' : 'comment to post to the target issue',
'required' : True,
}),
]
def get_query(args): def get_query(args):
@ -26,20 +35,6 @@ def main(args, stream):
response = graphql(get_query(args)) response = graphql(get_query(args))
stream.writeline(response) stream.writeline(response)
##################################################################### #####################################################################
execute(main, if __name__ == '__main__':
description = 'comment on an inssue in linear.app', execute(main, description, parse_args)
parse_args = [
( ['-d', '--issue-id'], {
'dest' : 'issue_id',
'help' : 'issue short-code (e.g. CLOUD-319)',
'required' : True,
}),
( ['-m', '--message'], {
'dest' : 'message',
'help' : 'comment to post to the target issue',
'required' : True,
}),
]
)

21
py/redis/interactive.py
View File

@ -1,15 +1,14 @@
#!/usr/bin/env python #!/usr/bin/env python
from py.lib.redis import get_client from scwrypts import execute
from py.lib.scwrypts import execute, interactive, getenv
from py.lib.scwrypts.exceptions import ImportedExecutableError
if __name__ != '__main__':
raise ImportedExecutableError()
##################################################################### #####################################################################
from scwrypts import interactive
from scwrypts.env import getenv
from scwrypts.redis import get_client
description = 'establishes a redis client in an interactive python shell'
parse_args = []
@interactive([ @interactive([
f'r = StrictRedis(\'{getenv("REDIS_HOST")}:{getenv("REDIS_PORT")}\')', f'r = StrictRedis(\'{getenv("REDIS_HOST")}:{getenv("REDIS_PORT")}\')',
]) ])
@ -20,7 +19,5 @@ def main(_args, _stream):
##################################################################### #####################################################################
execute(main, if __name__ == '__main__':
description = 'establishes a redis client in an interactive python shell', execute(main, description, parse_args)
parse_args = [],
)

6
py/requirements.txt
View File

@ -1,5 +1 @@
bpython ./lib
pyfzf
pyyaml
redis
twilio

67
py/twilio/send-sms.py
View File

@ -1,15 +1,38 @@
#!/usr/bin/env python #!/usr/bin/env python
from scwrypts import execute
#####################################################################
from sys import stderr from sys import stderr
from py.lib.scwrypts import execute, getenv from scwrypts.env import getenv
from py.lib.twilio import send_sms from scwrypts.twilio import send_sms
from py.lib.scwrypts.exceptions import ImportedExecutableError, MissingFlagAndEnvironmentVariableError
if __name__ != '__main__': description = 'send a simple SMS through twilio'
raise ImportedExecutableError() parse_args = [
( ['-t', '--to'], {
##################################################################### 'dest' : 'to',
'help' : 'phone number of the receipient',
'required' : False,
'default' : getenv('TWILIO__DEFAULT_PHONE_TO', required=False),
}),
( ['-f', '--from'], {
'dest' : 'from_',
'help' : 'phone number of the receipient',
'required' : False,
'default' : getenv('TWILIO__DEFAULT_PHONE_FROM', required=False),
}),
( ['-b', '--body'], {
'dest' : 'body',
'help' : 'message body',
'required' : False,
}),
( ['--max-char-count'], {
'dest' : 'max_char_count',
'help' : 'separate message into parts by character count (1 < N <= 1500)',
'required' : False,
'default' : 300,
}),
]
def main(args, stream): def main(args, stream):
if args.body is None: if args.body is None:
@ -35,31 +58,5 @@ def main(args, stream):
##################################################################### #####################################################################
execute(main, if __name__ == '__main__':
description = 'send a simple SMS through twilio', execute(main, description, parse_args)
parse_args = [
( ['-t', '--to'], {
'dest' : 'to',
'help' : 'phone number of the receipient',
'required' : False,
'default' : getenv('TWILIO__DEFAULT_PHONE_TO', required=False),
}),
( ['-f', '--from'], {
'dest' : 'from_',
'help' : 'phone number of the receipient',
'required' : False,
'default' : getenv('TWILIO__DEFAULT_PHONE_FROM', required=False),
}),
( ['-b', '--body'], {
'dest' : 'body',
'help' : 'message body',
'required' : False,
}),
( ['--max-char-count'], {
'dest' : 'max_char_count',
'help' : 'separate message into parts by character count (1 < N <= 1500)',
'required' : False,
'default' : 300,
}),
]
)

341
run
View File

@ -1,12 +1,13 @@
#!/bin/zsh #!/bin/zsh
export EXECUTION_DIR=$(pwd) export EXECUTION_DIR=$(pwd)
source "${0:a:h}/zsh/lib/import.driver.zsh" || exit 42 source "${0:a:h}/zsh/lib/import.driver.zsh" || exit 42
##################################################################### #####################################################################
() {
__RUN() { cd "$SCWRYPTS_ROOT__scwrypts"
GIT_SCWRYPTS() { git -C "$SCWRYPTS_ROOT__scwrypts" $@; }
local ERRORS=0
local USAGE=' local USAGE='
usage: scwrypts [... options ...] [patterns] -- [...script options...] usage: scwrypts [...options...] [...patterns...] -- [...script options...]
options: options:
selection selection
@ -18,9 +19,13 @@ __RUN() {
runtime runtime
-y, --yes auto-accept all [yn] prompts through current scwrypt -y, --yes auto-accept all [yn] prompts through current scwrypt
-e, --env <env-name> set environment; overwrites SCWRYPTS_ENV -e, --env <env-name> set environment; overwrites SCWRYPTS_ENV
-q, --quiet run in quiet mode -n shorthand for "--log-level 0"
-n, --no-log skip the log file and run in quiet mode -v, --log-level [0-4] set scwrypts log level to one of the following:
-v, --verbose override quiet mode settings and print all debug dialogue 0 : only command output and critical failures; skips logfile
1 : add success / failure messages
2 : (default) include status update messages
3 : (CI default) include warning messages
4 : include debug messages
alternate commands alternate commands
-h, --help display this message and exit -h, --help display this message and exit
@ -28,27 +33,28 @@ __RUN() {
--list-envs print out environment list and exit --list-envs print out environment list and exit
--update update scwrypts library to latest version --update update scwrypts library to latest version
--version print out scwrypts version and exit --version print out scwrypts version and exit
patterns: patterns:
- a list of glob patterns to loose-match a scwrypt by name - a list of glob patterns to loose-match a scwrypt by name
script options: script options:
- everything after "--" is forwarded to the scwrypt you run - everything after "--" is forwarded to the scwrypt you run
(usually "-- --help" will provide more information) ("-- --help" will provide more information)
' '
cd "$SCWRYPTS_ROOT"
#####################################################################
### cli argument parsing and global configuration ###################
#####################################################################
local ENV_NAME="$SCWRYPTS_ENV" local ENV_NAME="$SCWRYPTS_ENV"
local SEARCH_PATTERNS=() local SEARCH_PATTERNS=()
local VARSPLIT SEARCH_GROUP SEARCH_TYPE SEARCH_NAME local VARSPLIT SEARCH_GROUP SEARCH_TYPE SEARCH_NAME
local ALLOW_LOGFILE=1 [ ! $SCWRYPTS_LOG_LEVEL ] && {
local VERBOSE=1 local SCWRYPTS_LOG_LEVEL
[ $CI ] && SCWRYPTS_LOG_LEVEL=3 || SCWRYPTS_LOG_LEVEL=2
[ $CI ] && [ ! $SCWRYPTS_CI_FORCE_NON_VERBOSE ] && VERBOSE=2 }
local ERROR=0
while [[ $# -gt 0 ]] while [[ $# -gt 0 ]]
do do
@ -58,30 +64,44 @@ __RUN() {
set -- $(echo " $VARSPLIT ") ${@:2} set -- $(echo " $VARSPLIT ") ${@:2}
;; ;;
-h | --help ) USAGE; return 0 ;; ### alternate commands ###################
-l | --list ) SCWRYPTS__GET_AVAILABLE_SCWRYPTS; return 0 ;;
--list-envs ) SCWRYPTS__GET_ENV_NAMES; return 0 ;; -h | --help )
USAGE
return 0
;;
-l | --list )
SCWRYPTS__GET_AVAILABLE_SCWRYPTS
return 0
;;
--list-envs )
SCWRYPTS__GET_ENV_NAMES
return 0
;;
--version ) --version )
echo scwrypts $(git -C "$SCWRYPTS__ROOT__scwrypts" describe --tags) echo scwrypts $(GIT_SCWRYPTS describe --tags)
return 0 return 0
;; ;;
--update ) --update )
cd "$SCWRYPTS__ROOT__scwrypts" GIT_SCWRYPTS fetch --quiet origin main
git fetch --quiet origin main GIT_SCWRYPTS fetch --quiet origin main --tags
git fetch --quiet origin main --tags
local SYNC_STATUS=$? local SYNC_STATUS=$?
git diff --exit-code origin/main -- . >&2 GIT_SCWRYPTS diff --exit-code origin/main -- . >/dev/null 2>&1
local DIFF_STATUS=$? local DIFF_STATUS=$?
[[ $SYNC_STATUS -eq 0 ]] && [[ $DIFF_STATUS -eq 0 ]] && { [[ $SYNC_STATUS -eq 0 ]] && [[ $DIFF_STATUS -eq 0 ]] && {
SUCCESS 'already up-to-date with origin/main' SUCCESS 'already up-to-date with origin/main'
} || { } || {
git rebase --autostash origin/main \ GIT_SCWRYPTS rebase --autostash origin/main \
&& SUCCESS 'up-to-date with origin/main' \ && SUCCESS 'up-to-date with origin/main' \
&& GIT_SCWRYPTS log -n1 \
|| { || {
git rebase --abort GIT_SCWRYPTS rebase --abort
ERROR 'unable to update scwrypts; please try manual upgrade' ERROR 'unable to update scwrypts; please try manual upgrade'
REMINDER "installation in '$(pwd)'" REMINDER "installation in '$(pwd)'"
} }
@ -89,41 +109,52 @@ __RUN() {
return 0 return 0
;; ;;
### scwrypts filters #####################
-m | --name ) -m | --name )
[ ! $2 ] && ERROR "missing value for argument $1" && break [ $2 ] || { ERROR "missing value for argument $1"; break; }
SEARCH_NAME=$2 SEARCH_NAME=$2
shift 1 shift 1
;; ;;
-g | --group ) -g | --group )
[ ! $2 ] && ERROR "missing value for argument $1" && break [ $2 ] || { ERROR "missing value for argument $1"; break; }
SEARCH_GROUP=$2 SEARCH_GROUP=$2
shift 1 shift 1
;; ;;
-t | --type ) -t | --type )
[ ! $2 ] && ERROR "missing value for argument $1" && break [ $2 ] || { ERROR "missing value for argument $1"; break; }
SEARCH_TYPE=$2 SEARCH_TYPE=$2
shift 1 shift 1
;; ;;
-y | --yes ) export __SCWRYPTS_YES=1 ;; ### runtime settings #####################
-q | --quiet ) VERBOSE=0 ;;
-n | --no-log ) VERBOSE=0 ; [ ! $SUBSCWRYPT ] && SUBSCWRYPT=0 ;; -y | --yes ) export __SCWRYPTS_YES=1 ;;
-v | --verbose ) VERBOSE=2 ;;
-n | --no-log )
SCWRYPTS_LOG_LEVEL=0
[[ $1 =~ ^--no-log$ ]] && WARNING 'the --no-log flag is deprecated and will be removed in scwrypts v4.2'
;;
-v | --log-level )
[[ $2 =~ ^[0-4]$ ]] || ERROR "invalid setting for log-level '$2'"
SCWRYPTS_LOG_LEVEL=$2
shift 1
;;
-e | --env ) -e | --env )
[ ! $2 ] && ERROR "missing value for argument $1" && break [ $2 ] || { ERROR "missing value for argument $1"; break; }
[ ! $SUBSCWRYPTS ] \
&& [ $ENV_NAME ] \ [ $ENV_NAME ] && DEBUG 'overwriting session environment'
&& WARNING 'overwriting session environment' \
;
ENV_NAME="$2" ENV_NAME="$2"
STATUS "using CLI environment '$ENV_NAME'" STATUS "using CLI environment '$ENV_NAME'"
shift 1 shift 1
;; ;;
##########################################
-- ) shift 1; break ;; # pass arguments after '--' to the scwrypt -- ) shift 1; break ;; # pass arguments after '--' to the scwrypt
--* ) ERROR "unrecognized argument '$1'" ;; --* ) ERROR "unrecognized argument '$1'" ;;
@ -139,26 +170,20 @@ __RUN() {
CHECK_ERRORS CHECK_ERRORS
########################################## #####################################################################
### scwrypts selection / filtering ##################################
#####################################################################
local SCWRYPTS_AVAILABLE local SCWRYPTS_AVAILABLE
local POTENTIAL_ERROR="no such scwrypt exists:"
SCWRYPTS_AVAILABLE=$(SCWRYPTS__GET_AVAILABLE_SCWRYPTS) SCWRYPTS_AVAILABLE=$(SCWRYPTS__GET_AVAILABLE_SCWRYPTS)
[ $SEARCH_NAME ] && { ##########################################
POTENTIAL_ERROR+="\n NAME : '$SEARCH_NAME'"
POTENTIAL_ERROR+="\n TYPE : '$SEARCH_TYPE'"
POTENTIAL_ERROR+="\n GROUP : '$SEARCH_GROUP'"
SCWRYPTS_AVAILABLE=$({
echo $SCWRYPTS_AVAILABLE | head -n1
echo $SCWRYPTS_AVAILABLE | sed -e 's/\x1b\[[0-9;]*m//g' | grep "^$SEARCH_NAME *$SEARCH_TYPE *$SEARCH_GROUP\$"
})
}
[ ! $SEARCH_NAME ] && { [ $SEARCH_NAME ] && SCWRYPTS_AVAILABLE=$({
echo $SCWRYPTS_AVAILABLE | head -n1
echo $SCWRYPTS_AVAILABLE | sed -e 's/\x1b\[[0-9;]*m//g' | grep "^$SEARCH_NAME *$SEARCH_TYPE *$SEARCH_GROUP\$"
}) || {
[ $SEARCH_TYPE ] && { [ $SEARCH_TYPE ] && {
POTENTIAL_ERROR+="\n TYPE : '$SEARCH_TYPE'"
SCWRYPTS_AVAILABLE=$(\ SCWRYPTS_AVAILABLE=$(\
{ {
echo $SCWRYPTS_AVAILABLE | head -n1 echo $SCWRYPTS_AVAILABLE | head -n1
@ -171,7 +196,6 @@ __RUN() {
} }
[ $SEARCH_GROUP ] && { [ $SEARCH_GROUP ] && {
POTENTIAL_ERROR+="\n GROUP : '$SEARCH_GROUP'"
SCWRYPTS_AVAILABLE=$( SCWRYPTS_AVAILABLE=$(
{ {
echo $SCWRYPTS_AVAILABLE | head -n1 echo $SCWRYPTS_AVAILABLE | head -n1
@ -198,41 +222,47 @@ __RUN() {
} }
} }
[[ $(echo $SCWRYPTS_AVAILABLE | wc -l) -lt 2 ]] && ERROR "$POTENTIAL_ERROR" [[ $(echo $SCWRYPTS_AVAILABLE | wc -l) -lt 2 ]] && {
FAIL 1 "$(echo "
CHECK_ERRORS no such scwrypt exists
NAME : '$SEARCH_NAME'
TYPE : '$SEARCH_TYPE'
GROUP : '$SEARCH_GROUP'
PATTERNS : '$SEARCH_PATTERNS'
" | sed "1d; \$d; /''$/d")"
}
########################################## ##########################################
local NAME="$SEARCH_NAME" [[ $(echo $SCWRYPTS_AVAILABLE | wc -l) -eq 2 ]] \
local TYPE="$SEARCH_TYPE" && SCWRYPT_SELECTION=$(echo $SCWRYPTS_AVAILABLE | tail -n1) \
local GROUP="$SEARCH_GROUP" || SCWRYPT_SELECTION=$(echo $SCWRYPTS_AVAILABLE | FZF "select a script to run" --header-lines 1) \
;
[[ $(echo $SCWRYPTS_AVAILABLE | wc -l) -eq 2 ]] && {
SCWRYPT_SELECTION=$(echo $SCWRYPTS_AVAILABLE | tail -n1)
[[ $VERBOSE -eq 2 ]] || VERBOSE=0
} || {
SCWRYPT_SELECTION=$(echo $SCWRYPTS_AVAILABLE | FZF "select a script to run" --header-lines 1)
}
[ $SCWRYPT_SELECTION ] || exit 2 [ $SCWRYPT_SELECTION ] || exit 2
##########################################
local NAME TYPE GROUP
SCWRYPTS__SEPARATE_SCWRYPT_SELECTION $SCWRYPT_SELECTION SCWRYPTS__SEPARATE_SCWRYPT_SELECTION $SCWRYPT_SELECTION
export SCWRYPT_NAME=$NAME export SCWRYPT_NAME=$NAME
export SCWRYPT_TYPE=$TYPE export SCWRYPT_TYPE=$TYPE
export SCWRYPT_GROUP=$GROUP export SCWRYPT_GROUP=$GROUP
########################################## #####################################################################
### environment variables and configuration validation ##############
#####################################################################
local ENV_REQUIRED=true \
&& [ ! $CI ] \
&& [[ ! $SCWRYPT_NAME =~ scwrypts/logs ]] \
&& [[ ! $SCWRYPT_NAME =~ scwrypts/environment ]] \
|| ENV_REQUIRED=false
local ENV_REQUIRED=$(__CHECK_ENV_REQUIRED && echo 1 || echo 0)
local REQUIRED_ENVIRONMENT_REGEX=$(eval echo '$SCWRYPTS_REQUIRED_ENVIRONMENT_REGEX__'$SCWRYPT_GROUP) local REQUIRED_ENVIRONMENT_REGEX=$(eval echo '$SCWRYPTS_REQUIRED_ENVIRONMENT_REGEX__'$SCWRYPT_GROUP)
[ $REQUIRED_ENVIRONMENT_REGEX ] && { [[ $ENV_REQUIRED =~ true ]] && {
[[ $ENV_NAME =~ $REQUIRED_ENVIRONMENT_REGEX ]] \
|| FAIL 5 "group '$SCWRYPT_GROUP' requires current environment to match '$REQUIRED_ENVIRONMENT_REGEX' (currently $ENV_NAME)"
}
[[ $ENV_REQUIRED -eq 1 ]] && {
[ ! $ENV_NAME ] && ENV_NAME=$(SCWRYPTS__SELECT_ENV) [ ! $ENV_NAME ] && ENV_NAME=$(SCWRYPTS__SELECT_ENV)
for GROUP in ${SCWRYPTS_GROUPS[@]} for GROUP in ${SCWRYPTS_GROUPS[@]}
@ -249,105 +279,110 @@ __RUN() {
export ENV_NAME export ENV_NAME
} }
##########################################
[ $REQUIRED_ENVIRONMENT_REGEX ] && { [ $REQUIRED_ENVIRONMENT_REGEX ] && {
[[ $ENV_NAME =~ $REQUIRED_ENVIRONMENT_REGEX ]] \ [[ $ENV_NAME =~ $REQUIRED_ENVIRONMENT_REGEX ]] \
|| FAIL 5 "group '$SCWRYPT_GROUP' requires current environment to match '$REQUIRED_ENVIRONMENT_REGEX' (currently $ENV_NAME)" || FAIL 5 "group '$SCWRYPT_GROUP' requires current environment name to match '$REQUIRED_ENVIRONMENT_REGEX' (currently $ENV_NAME)"
} }
########################################## ##########################################
: \ [ ! $SUBSCWRYPT ] && [[ $ENV_NAME =~ prod ]] && {
&& [ ! $SUBSCWRYPT ] \ STATUS "on '$ENV_NAME'; checking diff against origin/main"
&& [[ $ENV_NAME =~ prod ]] \
&& { __VALIDATE_UPSTREAM_TIMELINE || ABORT; } GIT_SCWRYPTS fetch --quiet origin main
local SYNC_STATUS=$?
GIT_SCWRYPTS diff --exit-code origin/main -- . >&2
local DIFF_STATUS=$?
[[ $SYNC_STATUS -eq 0 ]] && [[ $DIFF_STATUS -eq 0 ]] && {
SUCCESS 'up-to-date with origin/main'
} || {
SCWRYPTS_LOG_LEVEL=3 WARNING "you are trying to run in ${__BRIGHT_RED}production${__YELLOW} but $([[ $SYNC_STATUS -ne 0 ]] && echo 'I am unable to verify your scwrypts version')$([[ $DIFF_STATUS -ne 0 ]] && echo 'your scwrypts is out-of-date (diff listed above)')"
yN 'continue?' || {
REMINDER "you can use 'scwrypts --update' to quickly update scwrypts to latest"
ABORT
}
}
}
########################################## ##########################################
local RUN_STRING=$(SCWRYPTS__GET_RUNSTRING $SCWRYPT_NAME $SCWRYPT_TYPE $SCWRYPT_GROUP) local RUN_STRING=$(SCWRYPTS__GET_RUNSTRING $SCWRYPT_NAME $SCWRYPT_TYPE $SCWRYPT_GROUP)
[ ! $RUN_STRING ] && exit 3 [ "$RUN_STRING" ] || return 42
########################################## #####################################################################
### logging and pretty header/footer setup ##########################
#####################################################################
local LOGFILE=$(__GET_LOGFILE) local LOGFILE \
&& [[ $SCWRYPTS_LOG_LEVEL -gt 0 ]] \
&& [ ! $SUBSCWRYPT ] \
&& [[ ! $SCWRYPT_NAME =~ scwrypts/logs ]] \
&& [[ ! $SCWRYPT_NAME =~ interactive ]] \
&& LOGFILE="$SCWRYPTS_LOG_PATH/$(echo $GROUP/$TYPE/$NAME | sed 's/^\.\///; s/\//\%/g').log" \
|| LOGFILE='/dev/null' \
;
local HEADER=$( local RUN_MODE=normal
[[ $VERBOSE -gt 0 ]] || return 0 [[ $LOGFILE =~ ^/dev/null$ ]] && RUN_MODE=no-logfile
[ $SUBSCWRYPT ] && return 0 [[ $SCWRYPT_NAME =~ interactive ]] && RUN_MODE=interactive
echo '====================================================================='
echo "script : $SCWRYPT_GROUP $SCWRYPT_TYPE $SCWRYPT_NAME"
echo "run at : $(date)"
echo "config : $ENV_NAME"
[ ! $LOGFILE ] && echo '\033[1;33m------------------------------------------\033[0m'
)
[ ! $LOGFILE ] && { local HEADER FOOTER
[ $HEADER ] && echo $HEADER
[ $SUBSCWRYPT ] && { [[ $SCWRYPTS_LOG_LEVEL -ge 2 ]] && {
eval "$RUN_STRING $(printf "%q " "$@")" HEADER=$(
exit $? echo "
} || { =====================================================================
eval "$RUN_STRING $(printf "%q " "$@")" </dev/tty >/dev/tty 2>&1 script : $SCWRYPT_GROUP $SCWRYPT_TYPE $SCWRYPT_NAME
exit $? run at : $(date)
} config : $ENV_NAME
log level : $SCWRYPTS_LOG_LEVEL
\\033[1;33m--- SCWRYPT BEGIN ---------------------------------------------------\\033[0m
" | sed 's/^\s\+//; 1d'
)
FOOTER="\\033[1;33m--- SCWRYPT END ---------------------------------------------------\\033[0m"
} }
[ $SUBSCWRYPT ] && {
HEADER="\\033[0;33m--- ($SUBSCWRYPT) BEGIN $SCWRYPT_GROUP $SCWRYPT_TYPE $SCWRYPT_NAME ---"
FOOTER="\\033[0;33m--- ($SUBSCWRYPT) END $SCWRYPT_GROUP $SCWRYPT_TYPE $SCWRYPT_NAME ---"
}
#####################################################################
### run the scwrypt #################################################
#####################################################################
[ ! $SUBSCWRYPT ] && export SUBSCWRYPT=0
set -o pipefail
{ {
[ $HEADER ] && echo $HEADER [ $HEADER ] && echo $HEADER
[[ $VERBOSE -gt 0 ]] && echo '\033[1;33m--- BEGIN OUTPUT -------------------------\033[0m' case $RUN_MODE in
(eval "$RUN_STRING $(printf "%q " "$@")") normal )
EXIT_CODE=$? (eval "$RUN_STRING $(printf "%q " "$@")")
[[ $VERBOSE -gt 0 ]] && echo '\033[1;33m--- END OUTPUT ---------------------------\033[0m' EXIT_CODE=$?
;;
no-logfile )
eval "$RUN_STRING $(printf "%q " "$@")"
EXIT_CODE=$?
;;
interactive )
eval "$RUN_STRING $(printf "%q " "$@")" </dev/tty >/dev/tty 2>&1
EXIT_CODE=$?
;;
esac
[ $FOOTER ] && echo $FOOTER
[[ $EXIT_CODE -eq 0 ]] && EXIT_COLOR='32m' || EXIT_COLOR='31m' [[ $EXIT_CODE -eq 0 ]] && EXIT_COLOR='32m' || EXIT_COLOR='31m'
[[ $VERBOSE -gt 0 ]] && echo "terminated with\\033[1;$EXIT_COLOR code $EXIT_CODE\\033[0m" [[ $SCWRYPTS_LOG_LEVEL -ge 2 ]] && [ ! $SUBSCWRYPT ] \
&& echo "terminated with\\033[1;$EXIT_COLOR code $EXIT_CODE\\033[0m"
return $EXIT_CODE
} 2>&1 | tee --append "$LOGFILE" } 2>&1 | tee --append "$LOGFILE"
exit $(\ } $@
sed -n 's/^terminated with.*code \([0-9]*\).*$/\1/p' $LOGFILE \
| tail -n1
)
}
#####################################################################
__CHECK_ENV_REQUIRED() {
[ $CI ] && return 1
echo $SCWRYPT_NAME | grep -q 'scwrypts/logs/' && return 1
echo $SCWRYPT_NAME | grep -q 'scwrypts/environment/' && return 1
return 0
}
__VALIDATE_UPSTREAM_TIMELINE() {
STATUS "on '$ENV_NAME'; checking diff against origin/main"
git fetch --quiet origin main
local SYNC_STATUS=$?
git diff --exit-code origin/main -- . >&2
local DIFF_STATUS=$?
[[ $SYNC_STATUS -eq 0 ]] && [[ $DIFF_STATUS -eq 0 ]] && {
SUCCESS 'up-to-date with origin/main'
} || {
WARNING
[[ $SYNC_STATUS -ne 0 ]] && WARNING 'unable to synchronize with origin/main'
[[ $DIFF_STATUS -ne 0 ]] && WARNING 'your branch differs from origin/main (diff listed above)'
WARNING
yN 'continue?' || return 1
}
}
__GET_LOGFILE() {
[ $SUBSCWRYPT ] && return 0
[[ $SCWRYPT_NAME =~ scwrypts/logs ]] && return 0
[[ $SCWRYPT_NAME =~ interactive ]] && return 0
echo "$SCWRYPTS_LOG_PATH/$(echo $GROUP/$TYPE/$NAME | sed 's/^\.\///; s/\//\%/g').log"
}
#####################################################################
__RUN $@

2
scwrypts.plugin.zsh
View File

@ -14,7 +14,7 @@ SCWRYPTS__ZSH_PLUGIN() {
which scwrypts >/dev/null 2>&1\ which scwrypts >/dev/null 2>&1\
&& RBUFFER="scwrypts" || RBUFFER="$SCWRYPTS_ROOT/scwrypts" && RBUFFER="scwrypts" || RBUFFER="$SCWRYPTS_ROOT/scwrypts"
RBUFFER+=" --name $NAME --group $GROUP --type $TYPE --verbose" RBUFFER+=" --name $NAME --group $GROUP --type $TYPE"
zle accept-line zle accept-line
} }

9
zsh/cloud/aws/ecr/login
View File

@ -1,10 +1,7 @@
#!/bin/zsh #!/bin/zsh
DEPENDENCIES+=()
REQUIRED_ENV+=()
use cloud/aws/ecr use cloud/aws/ecr
CHECK_ENVIRONMENT
##################################################################### #####################################################################
ECR_LOGIN $@ MAIN() {
ECR_LOGIN $@
}

7
zsh/cloud/aws/efs/mount
View File

@ -3,11 +3,9 @@ DEPENDENCIES+=(jq)
REQUIRED_ENV+=(AWS__EFS__LOCAL_MOUNT_POINT) REQUIRED_ENV+=(AWS__EFS__LOCAL_MOUNT_POINT)
use cloud/aws/cli use cloud/aws/cli
CHECK_ENVIRONMENT
##################################################################### #####################################################################
EFS_CONNECT() { MAIN() {
GETSUDO || exit 1 GETSUDO || exit 1
[ ! -d $AWS__EFS__LOCAL_MOUNT_POINT ] && { [ ! -d $AWS__EFS__LOCAL_MOUNT_POINT ] && {
sudo mkdir $AWS__EFS__LOCAL_MOUNT_POINT \ sudo mkdir $AWS__EFS__LOCAL_MOUNT_POINT \
@ -64,6 +62,3 @@ EFS_CONNECT() {
FAIL 2 "unable to mount '$FS_ID'" FAIL 2 "unable to mount '$FS_ID'"
} }
} }
#####################################################################
EFS_CONNECT $@

7
zsh/cloud/aws/efs/unmount
View File

@ -3,11 +3,9 @@ DEPENDENCIES+=(jq)
REQUIRED_ENV+=(AWS__EFS__LOCAL_MOUNT_POINT) REQUIRED_ENV+=(AWS__EFS__LOCAL_MOUNT_POINT)
use cloud/aws/cli use cloud/aws/cli
CHECK_ENVIRONMENT
##################################################################### #####################################################################
EFS_DISCONNECT() { MAIN() {
[ ! -d "$AWS__EFS__LOCAL_MOUNT_POINT" ] && { [ ! -d "$AWS__EFS__LOCAL_MOUNT_POINT" ] && {
STATUS 'no efs currently mounted' STATUS 'no efs currently mounted'
exit 0 exit 0
@ -32,6 +30,3 @@ EFS_DISCONNECT() {
&& SUCCESS "done" \ && SUCCESS "done" \
|| FAIL 2 "failed to unmount '$EFS'" || FAIL 2 "failed to unmount '$EFS'"
} }
#####################################################################
EFS_DISCONNECT $@

9
zsh/cloud/aws/eks/login
View File

@ -1,10 +1,7 @@
#!/bin/zsh #!/bin/zsh
DEPENDENCIES+=()
REQUIRED_ENV+=()
use cloud/aws/eks use cloud/aws/eks
CHECK_ENVIRONMENT
##################################################################### #####################################################################
EKS__CLUSTER_LOGIN $@ MAIN() {
EKS__CLUSTER_LOGIN $@
}

10
zsh/cloud/aws/rds/create-backup
View File

@ -1,19 +1,11 @@
#!/bin/zsh #!/bin/zsh
DEPENDENCIES+=()
REQUIRED_ENV+=()
use cloud/aws/rds use cloud/aws/rds
use db/postgres use db/postgres
CHECK_ENVIRONMENT
##################################################################### #####################################################################
CREATE_BACKUP() { MAIN() {
local DB_HOST DB_PORT DB_NAME DB_USER DB_PASS local DB_HOST DB_PORT DB_NAME DB_USER DB_PASS
RDS__GET_DATABASE_CREDENTIALS $@ || return 1 RDS__GET_DATABASE_CREDENTIALS $@ || return 1
PG_DUMP PG_DUMP
} }
#####################################################################
CREATE_BACKUP $@

11
zsh/cloud/aws/rds/interactive-login
View File

@ -1,20 +1,11 @@
#!/bin/zsh #!/bin/zsh
DEPENDENCIES+=()
REQUIRED_ENV+=()
use cloud/aws/rds use cloud/aws/rds
use db/postgres use db/postgres
CHECK_ENVIRONMENT
##################################################################### #####################################################################
RDS_INTERACTIVE_LOGIN() { MAIN() {
local DB_HOST DB_PORT DB_NAME DB_USER DB_PASS local DB_HOST DB_PORT DB_NAME DB_USER DB_PASS
RDS__GET_DATABASE_CREDENTIALS $@ || return 1 RDS__GET_DATABASE_CREDENTIALS $@ || return 1
POSTGRES__LOGIN_INTERACTIVE POSTGRES__LOGIN_INTERACTIVE
} }
#####################################################################
RDS_INTERACTIVE_LOGIN $@

11
zsh/cloud/aws/rds/load-backup
View File

@ -1,20 +1,11 @@
#!/bin/zsh #!/bin/zsh
DEPENDENCIES+=()
REQUIRED_ENV+=()
use cloud/aws/rds use cloud/aws/rds
use db/postgres use db/postgres
CHECK_ENVIRONMENT
##################################################################### #####################################################################
LOAD_BACKUP() { MAIN() {
local DB_HOST DB_PORT DB_NAME DB_USER DB_PASS local DB_HOST DB_PORT DB_NAME DB_USER DB_PASS
RDS__GET_DATABASE_CREDENTIALS $@ || return 1 RDS__GET_DATABASE_CREDENTIALS $@ || return 1
PG_RESTORE PG_RESTORE
} }
#####################################################################
LOAD_BACKUP $@

9
zsh/cloud/aws/route53/backup
View File

@ -1,11 +1,9 @@
#!/bin/zsh #!/bin/zsh
DEPENDENCIES+=(cli53) DEPENDENCIES+=(cli53)
REQUIRED_ENV+=(AWS_PROFILE) REQUIRED_ENV+=(AWS_PROFILE)
CHECK_ENVIRONMENT
##################################################################### #####################################################################
ROUTE53_BACKUP() { MAIN() {
local BACKUP_PATH="$SCWRYPTS_OUTPUT_PATH/$ENV_NAME/aws-dns-backup/$(date '+%Y-%m-%d')" local BACKUP_PATH="$SCWRYPTS_OUTPUT_PATH/$ENV_NAME/aws-dns-backup/$(date '+%Y-%m-%d')"
mkdir -p $BACKUP_PATH >/dev/null 2>&1 mkdir -p $BACKUP_PATH >/dev/null 2>&1
@ -25,12 +23,11 @@ ROUTE53_BACKUP() {
for P in ${JOBS[@]}; do wait $P >/dev/null 2>&1; done for P in ${JOBS[@]}; do wait $P >/dev/null 2>&1; done
} }
#####################################################################
ROUTE53_GET_DOMAINS() { ROUTE53_GET_DOMAINS() {
cli53 list --profile $AWS_PROFILE \ cli53 list --profile $AWS_PROFILE \
| awk '{print $2;}' \ | awk '{print $2;}' \
| sed '1d; s/\.$//'\ | sed '1d; s/\.$//'\
; ;
} }
#####################################################################
ROUTE53_BACKUP

9
zsh/cloud/media-sync/pull
View File

@ -1,10 +1,7 @@
#!/bin/zsh #!/bin/zsh
DEPENDENCIES+=()
REQUIRED_ENV+=()
use cloud/media-sync use cloud/media-sync
CHECK_ENVIRONMENT
##################################################################### #####################################################################
MEDIA_SYNC__PULL $@ MAIN() {
MEDIA_SYNC__PULL $@
}

Some files were not shown because too many files have changed in this diff Show More