Compare commits

..

13 commits

Author SHA1 Message Date
b0b3c3d3db
chore: bump version to 0.1.8
Updates project version from 0.1.7 to 0.1.8 for release
2024-12-06 08:35:50 +01:00
de15cbdb75
feat: Enhances wiki article info fetching
Refactors to integrate fetching article info, badges, and interwiki links in a single request. Adds support to display category members and improves interwiki link processing by translating links where feasible. Adds error handling for interwiki processing. Updates HTML template to display category members.

Improves performance and reduces external requests, enhancing user navigation and data retrieval efficiency.

Fixes #19.
2024-12-06 08:35:28 +01:00
4479a109b3
fix: Refines badge image styling
Adjusts CSS selector for badge images to enhance layout control.
Ensures badge images maintain consistent size across different elements.
2024-12-06 08:22:23 +01:00
c702671243
chore: bump project version to 0.1.7
Updates the version from 0.1.6 to 0.1.7 to reflect recent changes or improvements made in the project.
2024-12-05 17:46:11 +01:00
2167528f19
feat: Add badge support to wiki articles
Fetches and displays badges for articles using the MediaWiki API.
Updates templates to show badges as icons linking to detailed info.
Adds CSS styling for badge presentation in the article view.

Enhances user experience by providing additional context through
quick-access badge information.

Fixes #21.
2024-12-05 17:45:47 +01:00
5bd90742b0
feat: Enhances readability with country flags
Adds emoji flags next to country names to improve
visual clarity and usability in the instances list.

Improves user understanding of instance locations.
2024-12-05 17:15:59 +01:00
5d35208968
chore: bump version to 0.1.6
Updates the project version from 0.1.5 to 0.1.6 in the pyproject.toml file. This version increment likely signifies minor changes, improvements, or bug fixes in the project.
2024-12-05 13:27:49 +01:00
cfc88a5c4e
feat: Enhances language sorting and interwiki feature
Adds functionality to sort languages by user activity with overriding via environment variables.
Implements fetching and integration of interwiki links using the MediaWiki API.
Introduces UI elements for language selection with new styling.

Improves user experience by prioritizing more active languages and providing easy navigation via interwiki links.

Fixes #45
2024-12-05 13:25:41 +01:00
01b63f7a82
feat: Enhances image loading with lazy loading
Adds lazy loading attribute to images to improve page load
performance and user experience by deferring the loading
of images until they are in the viewport.

Fixes #44.
2024-12-04 14:45:54 +01:00
7715094173
feat(README): Enhances documentation with new instances and guidance
Adds new public and Tor hidden service instances to the instances
section, including a new entry for Darkness.services.

Introduces sections for Tor Hidden Services and instructions
for adding new instances via pull requests or issues.

Enhances user support by detailing how to open issues or
provide feedback through Git or Github, and promotes
community engagement via a Matrix room.
2024-11-25 09:04:49 +01:00
28852e812c
Merge branch 'main' of git.private.coffee:privatecoffee/wikimore 2024-11-09 15:15:29 +01:00
d9a395ec76
docs: update README with new public instance
Added a new public instance of Wikimore hosted by Lumaeris to the README list. This helps users discover more available instances and expands the visibility of the project.

Closes #42.
2024-11-09 15:14:33 +01:00
a4e9eb37ad Merge pull request 'Add blitzw.in instance' (#41) from blitzwing/wikimore:blitzwing-patch-1 into main
Reviewed-on: PrivateCoffee/wikimore#41
2024-11-05 06:35:33 +00:00
5 changed files with 389 additions and 14 deletions

View file

@ -19,13 +19,30 @@ This project is still in development and more features will be added in the futu
## Instances ## Instances
| URL | Provided by | Country | Comments | | URL | Provided by | Country | Comments |
| ----------------------------------------------------------- | ----------------------------------------- | ------- | -------- | | ----------------------------------------------------------------- | ----------------------------------------------- | ------------- | -------- |
| [wikimore.private.coffee](https://wikimore.private.coffee/) | [Private.coffee](https://private.coffee/) | Austria | | | [wikimore.private.coffee](https://wikimore.private.coffee/) | [Private.coffee](https://private.coffee/) | Austria 🇦🇹 🇪🇺 | |
| [wm.bloat.cat](https://wm.bloat.cat/) | [bloat.cat](https://bloat.cat/) | Germany | | | [wm.bloat.cat](https://wm.bloat.cat/) | [bloat.cat](https://bloat.cat/) | Germany 🇩🇪 🇪🇺 | |
| [wikimore.blitzw.in](https://wikimore.blitzw.in/) | [blitzw.in](https://blitzw.in/) | Denmark | | | [wikimore.blitzw.in](https://wikimore.blitzw.in/) | [blitzw.in](https://blitzw.in/) | Denmark 🇩🇰 🇪🇺 | |
| [wikimore.lumaeris.com](https://wikimore.lumaeris.com/) | [Lumaeris](https://lumaeris.com/) | Germany 🇩🇪 🇪🇺 | |
| [wikimore.darkness.services](https://wikimore.darkness.services/) | [Darkness.services](https://darkness.services/) | USA 🇺🇸 | |
If you operate a public instance of Wikimore and would like to have it listed here, please open an issue or a pull request. ### Tor Hidden Services
| URL | Provided by | Country | Comments |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------- | ------------- | -------- |
| [wikimore.coffee2m3bjsrrqqycx6ghkxrnejl2q6nl7pjw2j4clchjj6uk5zozad.onion](http://wikimore.coffee2m3bjsrrqqycx6ghkxrnejl2q6nl7pjw2j4clchjj6uk5zozad.onion/) | [Private.coffee](https://private.coffee/) | Austria 🇦🇹 🇪🇺 | |
| [wikimore.darknessrdor43qkl2ngwitj72zdavfz2cead4t5ed72bybgauww5lyd.onion](http://wikimore.darknessrdor43qkl2ngwitj72zdavfz2cead4t5ed72bybgauww5lyd.onion/) | [Darkness.services](https://darkness.services/) | USA 🇺🇸 | |
### Adding Your Instance
To add your own instance to this list, please open a pull request or issue, see below.
## Opening Issues
If you're having problems using Wikimore, or if you have ideas or feedback for us, feel free to open an issue in the [Private.coffee Git](https://git.private.coffee/PrivateCoffee/wikimore/issues) or on [Github](https://github.com/PrivateCoffee/wikimore/issues).
Of course, you can also join our [Matrix room](https://matrix.pcof.fi/#/#wikimore:private.coffee) to discuss your ideas with us.
## Installation ## Installation

View file

@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project] [project]
name = "wikimore" name = "wikimore"
version = "0.1.5" version = "0.1.8"
authors = [{ name = "Private.coffee Team", email = "support@private.coffee" }] authors = [{ name = "Private.coffee Team", email = "support@private.coffee" }]
description = "A simple frontend for Wikimedia wikis" description = "A simple frontend for Wikimedia wikis"
readme = "README.md" readme = "README.md"

View file

@ -123,6 +123,96 @@ logger.debug(
) )
# Get number of active Wikipedia users for each language
def get_active_users() -> Dict[str, int]:
"""Fetch the number of active Wikipedia users for each language.
Returns:
Dict[str, int]: A dictionary mapping language codes to the number of active Wikipedia users.
"""
path = "/w/api.php?action=query&format=json&meta=siteinfo&siprop=statistics"
active_users = {}
for lang, data in app.languages.items():
try:
url = f"{data['projects']['wiki']}{path}"
with urllib.request.urlopen(url) as response:
data = json.loads(response.read().decode())
active_users[lang] = data["query"]["statistics"]["activeusers"]
except Exception as e:
logger.error(f"Error fetching active users for {lang}: {e}")
return sorted(active_users.items(), key=lambda x: x[1], reverse=True)
if os.environ.get("NO_LANGSORT", False):
LANGSORT = []
elif os.environ.get("LANGSORT") == "auto":
LANGSORT = [lang for lang, _ in get_active_users()[:50]]
elif os.environ.get("LANGSORT"):
LANGSORT = os.environ["LANGSORT"].split(",")
else:
# Opinionated sorting of languages
LANGSORT = [
"en",
"es",
"ja",
"de",
"fr",
"zh",
"ru",
"it",
"pt",
"pl",
"nl",
"ar",
]
def langsort(input: list[dict], key: str = "lang") -> list[dict]:
"""Sorting of language data.
Sorts a list of dictionaries containing "lang" keys such that the most common languages are first.
Allows specifying a custom order using the `LANGSORT` environment variable.
Args:
input (list[dict]): A list of dictionaries containing "lang" keys.
Returns:
list[dict]: The sorted list of dictionaries.
"""
if not LANGSORT:
return input
output = []
for lang in LANGSORT:
for item in input:
if item[key] == lang:
output.append(item)
for item in input:
if item[key] not in LANGSORT:
output.append(item)
return output
logger.debug("Initialized language sort order")
app_languages = [
{"lang": lang, "name": data["name"]} for lang, data in app.languages.items()
]
app_languages = langsort(app_languages)
app.languages = {
lang: app.languages[lang] for lang in [lang["lang"] for lang in app_languages]
}
def render_template(*args, **kwargs) -> Text: def render_template(*args, **kwargs) -> Text:
"""A wrapper around Flask's `render_template` that adds the `languages` and `wikimedia_projects` context variables. """A wrapper around Flask's `render_template` that adds the `languages` and `wikimedia_projects` context variables.
@ -243,18 +333,16 @@ def inbound_redirect(domain: str, url: str) -> Union[Text, Response, Tuple[Text,
Returns: Returns:
Response: A redirect to the corresponding route Response: A redirect to the corresponding route
""" """
# TODO: Make this the default route scheme instead of a redirect
for language, language_projects in app.languages.items(): for language, language_projects in app.languages.items():
for project_name, project_url in language_projects["projects"].items(): for project_name, project_url in language_projects["projects"].items():
if project_url == f"https://{domain}": if project_url == f"https://{domain}":
return redirect( return redirect(f"{url_for('home')}{project_name}/{language}/{url}")
f"{url_for('home')}{project_name}/{language}/{url}"
)
for project_name, project_url in app.languages["special"]["projects"].items(): for project_name, project_url in app.languages["special"]["projects"].items():
if project_url == f"https://{domain}": if project_url == f"https://{domain}":
return redirect( return redirect(f"{url_for('home')}/{project_name}/{language}/{url}")
f"{url_for('home')}/{project_name}/{language}/{url}"
)
# TODO / IDEA: Handle non-Wikimedia Mediawiki projects here? # TODO / IDEA: Handle non-Wikimedia Mediawiki projects here?
@ -267,6 +355,7 @@ def inbound_redirect(domain: str, url: str) -> Union[Text, Response, Tuple[Text,
404, 404,
) )
@app.route("/<project>/<lang>/wiki/<path:title>") @app.route("/<project>/<lang>/wiki/<path:title>")
def wiki_article( def wiki_article(
project: str, lang: str, title: str project: str, lang: str, title: str
@ -303,11 +392,146 @@ def wiki_article(
logger.debug(f"Fetching {title} from {base_url}") logger.debug(f"Fetching {title} from {base_url}")
# Check if the article is something we need to handle differently
info_api_request = urllib.request.Request(
f"{base_url}/w/api.php?action=query&format=json&titles={escape(quote(title.replace(' ', '_')), True)}&prop=info|pageprops|categoryinfo|langlinks&lllimit=500",
headers=HEADERS,
)
category_members = []
interwiki = []
badges = []
with urllib.request.urlopen(info_api_request) as response:
logger.debug(
f"Tried to fetch info for {title} from {info_api_request.full_url}"
)
data = json.loads(response.read().decode())
page = data["query"]["pages"].popitem()[1]
langlinks = page.get("langlinks", [])
logger.debug(f"Original Interwiki links for {title}: {langlinks}")
# Get interwiki links and translate them to internal links where possible
for link in langlinks:
try:
interwiki_lang = link["lang"]
interwiki_title = link["*"]
logger.debug(
f"Generating interwiki link for: {interwiki_lang}.{project}/{interwiki_title}"
)
interwiki_url = url_for(
"wiki_article",
project=project,
lang=interwiki_lang,
title=interwiki_title,
)
link["url"] = interwiki_url
link["langname"] = app.languages[interwiki_lang]["name"]
interwiki.append(link)
except KeyError as e:
logger.error(
f"Error processing interwiki link for title {title} in language {lang}: {e}"
)
# Get badges (e.g. "Good Article", "Featured Article")
props = page.get("pageprops", {})
for prop in props:
if prop.startswith("wikibase-badge-"):
try:
badge_id = prop.replace("wikibase-badge-", "")
# Fetch the badge data from Wikidata
badge_request = urllib.request.Request(
f"https://www.wikidata.org/w/api.php?action=wbgetentities&format=json&ids={badge_id}&languages={lang}",
headers=HEADERS,
)
with urllib.request.urlopen(badge_request) as badge_response:
logger.debug(
f"Tried to fetch badge {badge_id} from {badge_request.full_url}"
)
badge_data = json.loads(badge_response.read().decode())
badge = badge_data["entities"][badge_id]["labels"][lang][
"value"
]
badge_image = badge_data["entities"][badge_id]["claims"]["P18"][
0
]["mainsnak"]["datavalue"]["value"]
badges.append(
{
"title": badge,
"url": f"https://www.wikidata.org/wiki/{badge_id}",
"image": get_proxy_url(
f"https://commons.wikimedia.org/wiki/Special:Redirect/file/{badge_image}"
),
}
)
except Exception as e:
logger.error(f"Error fetching badge {prop}: {e}")
# If the article is a category, fetch the category members
if "categoryinfo" in page:
category_api_url = f"{base_url}/w/api.php?action=query&format=json&list=categorymembers&cmtitle={escape(quote(title.replace(' ', '_')), True)}&cmlimit=500"
category_api_request = urllib.request.Request(
category_api_url,
headers=HEADERS,
)
all_members = []
with urllib.request.urlopen(category_api_request) as category_api_response:
logger.debug(
f"Tried to fetch category members for {title} from {category_api_request.full_url}"
)
data = json.loads(category_api_response.read().decode())
category_members = data["query"]["categorymembers"]
all_members += category_members
if "continue" in data:
continue_params = f"&cmcontinue={data['continue']['cmcontinue']}"
category_api_request = urllib.request.Request(
category_api_url + continue_params,
headers=HEADERS,
)
with urllib.request.urlopen(
category_api_request
) as category_api_response:
data = json.loads(category_api_response.read().decode())
all_members += data["query"]["categorymembers"]
category_members = all_members
for member in category_members:
member["url"] = url_for(
"wiki_article",
project=project,
lang=lang,
title=member["title"],
)
interwiki = langsort(interwiki)
# Prepare the API request to fetch the article content
api_request = urllib.request.Request( api_request = urllib.request.Request(
f"{base_url}/api/rest_v1/page/html/{escape(quote(title.replace(' ', '_')), True).replace('/', '%2F')}", f"{base_url}/api/rest_v1/page/html/{escape(quote(title.replace(' ', '_')), True).replace('/', '%2F')}",
headers=HEADERS, headers=HEADERS,
) )
logger.debug(f"Article content URL: {api_request.full_url}")
# Add the `variant` header if the `variant` query parameter is present # Add the `variant` header if the `variant` query parameter is present
# This is used to fetch articles in a specific script variant (https://www.mediawiki.org/wiki/Writing_systems/LanguageConverter) # This is used to fetch articles in a specific script variant (https://www.mediawiki.org/wiki/Writing_systems/LanguageConverter)
if request.args.get("variant", None): if request.args.get("variant", None):
@ -428,6 +652,9 @@ def wiki_article(
for img in soup.find_all("img"): for img in soup.find_all("img"):
img["src"] = get_proxy_url(img["src"]) img["src"] = get_proxy_url(img["src"])
# While we're at it, ensure that images are loaded lazily
img["loading"] = "lazy"
for source in soup.find_all("source"): for source in soup.find_all("source"):
source["src"] = get_proxy_url(source["src"]) source["src"] = get_proxy_url(source["src"])
@ -487,6 +714,9 @@ def wiki_article(
project=project, project=project,
rtl=rtl, rtl=rtl,
license=license, license=license,
interwiki=interwiki,
badges=badges,
category_members=category_members,
) )

View file

@ -1228,4 +1228,95 @@ Currently blocked by implementation of comments retrieval in the backend
.side-box-text { .side-box-text {
font-size: 0.8em; font-size: 0.8em;
} }
}
/* Language selector styling */
.language-selector {
position: relative;
display: inline-block;
}
.language-selector-toggle {
display: none;
}
.language-selector-label {
cursor: pointer;
display: inline-block;
padding: 0.5em;
border: 1px solid #ccc;
border-radius: 0.25em;
}
.language-selector-label-text {
display: inline-block;
margin-right: 0.5em;
}
.language-selector-label-icon {
display: inline-block;
width: 0;
height: 0;
border-left: 0.25em solid transparent;
border-right: 0.25em solid transparent;
border-top: 0.25em solid #333;
}
.language-selector-menu {
display: none;
position: absolute;
top: 100%;
left: 0;
z-index: 1;
background-color: #fff;
border: 1px solid #ccc;
border-radius: 0.25em;
}
.language-selector-toggle:checked + .language-selector-label + .language-selector-menu {
display: block;
}
.language-selector-list {
list-style-type: none;
margin: 0;
padding: 0;
}
.language-selector-item {
border-top: 1px solid #ccc;
}
.language-selector-link {
display: block;
padding: 0.5em;
text-decoration: none;
color: #333;
}
.language-selector-link:hover {
background-color: #f0f0f0;
}
.language-selector-link:active {
background-color: #e0e0e0;
}
.language-selector-link:focus {
outline: 1px dotted #333;
outline: 5px auto -webkit-focus-ring-color;
}
.language-selector-link:active,
.language-selector-link:focus {
outline: none;
}
.language-selector-link:active,
.language-selector-link:focus {
outline: none;
}
.badge > img {
max-height: 1em;
} }

View file

@ -1,8 +1,45 @@
{% extends "base.html" %} {% extends "base.html" %}
{% block content %} {% block content %}
<h1 class="title{% if rtl %} title-rtl{% endif %}">{{ title }}</h1> <h1 class="title{% if rtl %} title-rtl{% endif %}">
{{ title }}
{% if badges %}
<span class="badges">
{% for badge in badges %}
<a href="{{ badge.url }}" class="badge" title="{{ badge.title }}">
<img src="{{ badge.image }}" alt="{{ badge.title }}">
</a>
{% endfor %}
</span>
{% endif %}
</h1>
{% if interwiki %}
<div class="language-selector">
<input type="checkbox" id="language-selector-toggle" class="language-selector-toggle">
<label for="language-selector-toggle" class="language-selector-label">
<span class="language-selector-label-text">Language</span>
<span class="language-selector-label-icon"></span>
</label>
<div class="language-selector-menu">
<ul class="language-selector-list">
{% for lang in interwiki %}
<li class="language-selector-item">
<a href="{{ lang.url }}" class="language-selector-link">{{ lang.langname }}</a>
</li>
{% endfor %}
</ul>
</div>
</div>
{% endif %}
{{ content|safe }} {{ content|safe }}
{% if category_members %}
<h2>Pages in category "{{ title }}"</h2>
<ul>
{% for member in category_members %}
<li><a href="{{ member.url }}">{{ member.title }}</a></li>
{% endfor %}
</ul>
{% endif %}
{% endblock %} {% endblock %}
{% block license %} {% block license %}