-
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Remove server functionality from media-parser
This commit removes all server-related functionalities from the media-parser package. As a result, the package now solely exists for parsing media from social networks. This creates a cleaner codebase and focuses on the core functionality of the package. Signed-off-by: Jag_k <[email protected]>
- Loading branch information
Showing
45 changed files
with
1,515 additions
and
3,046 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
ko_fi: jag_k | ||
github: jag-k |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -10,45 +10,17 @@ jobs: | |
publish: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v2 | ||
|
||
- uses: actions/setup-python@v2 | ||
with: | ||
python-version: "3.11" | ||
|
||
- name: Install poetry | ||
uses: abatilo/[email protected] | ||
- uses: actions/checkout@v4 | ||
- name: Install Poetry | ||
run: pipx install poetry==1.8.2 | ||
- uses: actions/setup-python@v5 | ||
with: | ||
poetry-version: "1.6.1" | ||
python-version: "3.12" | ||
cache: "poetry" | ||
|
||
- name: Publish to PyPI | ||
env: | ||
PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }} | ||
run: | | ||
poetry config pypi-token.pypi $PYPI_TOKEN | ||
poetry publish --build --no-interaction && echo "published=1" >> "$GITHUB_ENV" || echo "published=0" >> "$GITHUB_ENV" | ||
- name: Check if published | ||
run: | | ||
if [ "$published" = "1" ]; then | ||
echo "Published to PyPI" | ||
else | ||
echo "Failed to publish to PyPI" | ||
exit 1 | ||
fi | ||
# - name: Build Dash Docset | ||
# run: | | ||
# poetry run python -m dash_docs.tools.generate_docset | ||
|
||
- name: Make release with docset | ||
uses: softprops/action-gh-release@v1 | ||
with: | ||
# files: | | ||
# dash-docset.tgz | ||
tag_name: ${{ github.ref }} | ||
# body: | | ||
# Dash Docset for Dash ${{ github.ref }} | ||
draft: false | ||
prerelease: false | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -15,129 +15,15 @@ Server for parse Media by URL. | |
- [x] Instagram | ||
- [x] Twitter | ||
- [x] Reddit | ||
- [ ] VK | ||
|
||
## Installation and Configuration Server | ||
|
||
Use the `docker-compose.yml` file to run the server. | ||
|
||
```yaml | ||
version: "3.8" | ||
|
||
service: | ||
media-parser: | ||
image: ghcr.io/jag-k/media-parser:latest | ||
ports: | ||
- 8000:8000 | ||
environment: | ||
# Sentry integration (optional) | ||
SENTRY_DSN: "https://[email protected]/2" | ||
SENTRY_ENVIRONMENT: "dev" | ||
|
||
# Enable sentry user feedback (optional) | ||
SENTRY_ORGANISATION_SLUG: "sentry" | ||
SENTRY_PROJECT_SLUG: "media-parser" | ||
SENTRY_AUTH_TOKEN: "..." # with scope project:write | ||
SENTRY_API_HOST: "https://api.sentry.io/" | ||
|
||
# Database | ||
MONGO_URL: "mongodb://mongodb:27017" | ||
MONGO_DATABASE: "test" | ||
|
||
volumes: | ||
- ./config:/config | ||
|
||
mongodb: | ||
image: mongo:latest | ||
volumes: | ||
- ./data:/data/db | ||
``` | ||
### Parsers Configuration | ||
All configs for parsers stored in `config/parsers.json`. JSON Schema for | ||
this: [schemas/parser_schema.json](https://github.com/jag-k/media-parser/blob/main/schemas/parser_schema.json). | ||
|
||
To enable parser, you need to add config for this parser. | ||
If parser hasn't config, like `tiktok` set an empty object (`{}`) to enable it. | ||
|
||
Example: | ||
|
||
```json5 | ||
// config/parsers.json | ||
{ | ||
"$schema": "https://raw.github.com/jag-k/media-parsers/blob/main/schemas/parser_schema.json", | ||
"instagram": { | ||
// Optional | ||
"instagram_saas_token": "asdasd" | ||
}, | ||
"reddit": { | ||
"client_id": "", | ||
"client_secret": "", | ||
// Optional | ||
"user_agent": "video downloader (by u/Jag_k)" | ||
}, | ||
"tiktok": {}, | ||
"twitter": { | ||
"twitter_bearer_token": "asdasd" | ||
}, | ||
"youtube": {} | ||
} | ||
``` | ||
|
||
Or you can use YAML file like `config/parsers.yaml` or `config/parsers.yml`: | ||
|
||
```yaml | ||
# config/parsers.yml | ||
$schema: "https://raw.github.com/jag-k/media-parsers/blob/main/schemas/parser_schema.json" | ||
instagram: | ||
# Optional | ||
instagram_saas_token: "asdasd" | ||
reddit: | ||
client_id: "" | ||
client_secret: "" | ||
# Optional | ||
user_agent: "video downloader (by u/Jag_k)" | ||
tiktok: {} | ||
twitter: | ||
twitter_bearer_token: "asdasd" | ||
youtube: {} | ||
``` | ||
|
||
## Usage | ||
|
||
API documentation available on `/docs` endpoint. | ||
|
||
## Clients | ||
|
||
### Installation | ||
## Installation | ||
|
||
```bash | ||
poetry add media-parser # or pip install media-parser | ||
``` | ||
|
||
### Usage | ||
|
||
```python | ||
from media_parser import Client, FeedbackTypes | ||
client = Client(url="http://localhost:8000") | ||
async def main(): | ||
# Get all media | ||
media = await client.parse("https://www.youtube.com/watch?v=9bZkp7q19f0", user="jag-k") | ||
print(media) | ||
# If media is incorrect, you can send feedback | ||
await client.send_feedback(media, "jag-k", FeedbackTypes.wrong_media) | ||
if __name__ == '__main__': | ||
import asyncio | ||
asyncio.run(main()) | ||
``` | ||
|
||
## License | ||
|
||
|
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.