Commit 8e0004ee authored by Denis Salem's avatar Denis Salem
Browse files

wip on feed generation

parent 38960ce3
......@@ -16,10 +16,11 @@ markup_language: 'Markdown'
path:
category_directory_name: '{category}'
dates_directory_name: '%m-%Y'
entry_file_name: indexé.html
entry_file_name: index.html
ftp: Chemin absolu de votre blog sur l'hôte FTP.
index_file_name: index{page_number}.html
rss_file_name: feedé.xml
rss_file_name: rss_feed.xml
atom_file_name: atom_feed.xml
dates_sub_folders: ""
categories_sub_folders: ""
entries_sub_folders: "{entry_title}"
......@@ -32,7 +33,7 @@ disable_main_thread: true
disable_threads: ""
disable_archives: true
disable_categories: false
disable_single_entries: false
disable_single_entries: true
banner: 'https://download.tuxfamily.org/dsalem/img/2017_-_Denis_Salem_-_CC_By_SA_-_VenC-logo.svg'
path_encoding: 'windows-1252'
server_port: 8888
......@@ -20,42 +20,16 @@ Après cette commande, VenC a créer un répertoire appelé "MonSuperBlog" à l'
> .:SetColor::blog::blue:. blog_configuration.yaml .:SetColor::entries::blue:. .:SetColor::extra::blue:. .:SetColor::includes::blue:. .:SetColor::templates::blue:. .:SetColor::theme::blue:.
La prochaine étape consiste à paramétrer un peu le blog. Pour ça il suffit d'éditer le fichier *blog_configuration.yaml* à la racine de ce repertoire. Vous pourrez notamment y définir le nom de votre blog, sa langue, ses mots clef, etc.
.:CodeHighlight::Yaml::False::author_description: À propos de vous
author_email: Votre adresse email
author_name: Votre nom
blog_description: Courte description du blog
blog_keywords: Quelques mots clefs qui définissent votre blog
blog_language: La langue du blog
blog_name: Nom du blog
blog_url: L'url du blog
code_highlight_css_override: false
columns: 1
date_format: '%A %d. %B %Y'
disable_archives: false
disable_categories: false
disable_main_thread: false
disable_single_entries: false
disable_threads: ''
entries_per_pages: 10
ftp_host: Nom d'hôte FTP
license: La licence appliquée à votre contenu
markup_language: Markdown
path:
categories_sub_folders: ''
category_directory_name: '{category}'
dates_directory_name: '%Y-%m'
dates_sub_folders: ''
entries_sub_folders: ''
entry_file_name: entry{entry_id}.html
ftp: Chemin absolu de votre blog sur l'hôte FTP.
index_file_name: index{page_number}.html
rss_file_name: feed.xml
path_encoding: utf-8
reverse_thread_order: true
rss_thread_lenght: 5
text_editor: nano:.
La prochaine étape consiste à paramétrer un peu le blog. Pour ça il suffit d'éditer le fichier *blog_configuration.yaml* à la racine de ce repertoire. Vous pourrez notamment y définir le nom de votre blog, sa langue, ses mots clef, entre autre choses parmis les nombreux paramétres à configurer, ou laisser par défaut.
.:CodeHighlight::Yaml::False::author_description: 'Je suis Jean Rochefort et je suis mort, mais je reviendrais.'
author_name: 'Jean Fucking Rochefort'
blog_description: 'Les tribulations du roy de France Jean Rochefort en direct du paradis.'
blog_keywords: 'cinema,moustache,punchline,boloss'
blog_language: 'fr'
blog_name: 'Le blog de Jean Rochefort, élu roy de France par les internets.'
license: 'License royale':.
Pour plus d'informations sur la configuration du blog, rendez-vous [ici](https://framagit.org/denissalem/VenC/blob/master/doc/FR.md#fichier-de-configuration-principal)
Et voilà, c'est terminé! Votre blog est prêt à l'emploi!
......
<div id="entry-content">
<div class="entry" id="entry.:GetEntryID:." data-venc-keywords=".:ForEntryTags::{value}::,:." data-venc-authors=".:ForEntryAuthors::{author}::,:.">
<div class="entry" id="entry.:GetEntryID:." data-venc-keywords=".:ForEntryTags::{value}::,:." data-venc-authors=".:ForEntryAuthors::{value}::,:.">
<h2><a href=".:GetEntryURL:.">.:GetEntryTitle:.</a></h2>
.:GetEntryContent:.
</div>
......
......@@ -19,5 +19,6 @@
<h1><a href=".:GetRootPage:."><img src=".:GetRelativeOrigin:.logo.png" alt=".:GetBlogName:." title=".:GetBlogName:." /></a></h1>
</header>
.:TreeForBlogCategories::<ul>::<li><a href="{path}#entry-content">{value}</a>::</li>::</ul>:.
<ul>.:ForBlogDates::<li><a href="{path}">{value}</a></li>:.</ul>
</aside>
<div id="content">
<item>
<title>.:GetEntryTitle:.</title>
<link>.:GetBlogURL:./.:GetEntryURL:.</link>
<guid isPermaLink="true">.:GetBlogURL:./.:GetEntryURL:.</guid>
<description>.:GetEntryContent:.</description>
</item>
<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<atom:link href=".:GetBlogURL:./feed.xml" rel="self" type="application/rss+xml" />
<title>.:GetBlogName:.</title>
<link>.:GetBlogURL:.</link>
<description>.:GetBlogDescription:.</description>
......@@ -136,8 +136,10 @@
DONE | Support for migration.
DONE | Catch UnicodeEncodeError.
DONE | Serv page.
DONE | Replace python-markdown by markdown2
DONE | Replace python-markdown by markdown2.
DONE | Replace white spaces in url by dashes.
WIP | Add support for Atom feed.
TODO | Split unit-test and themes from main branch
TODO | Support ForPage in entries thread.
TODO | Blacklist user defined patterns from entry.
TODO | Update command help with.
......@@ -158,7 +160,6 @@
TODO | Add docutils reStructuredText parser settings override.
TODO | Decrease halt_level to render reStructuredText even if errors occurs.
TODO | Access {count} and {weight} from LeavesForEntrycategories.
TODO | Split unit-test and themes from main branch
TODO | Add Yaml comments in blog configuration
TODO | Catch invalid encoding at startup
TODO | Get formatted link to specified page/entry
......@@ -64,6 +64,6 @@ setup(name='VenC',
'pyyaml',
'Pygments'
],
scripts=['venc','venc-unit-test','venc-blog-migration'],
scripts=['venc','venc-blog-migration'],
data_files = extra_files
)
#! /usr/bin/python
# -*- coding: utf-8 -*-
import venc2.pattern
print("""#############################
# Testing Pattern Processor #
#############################\n""")
def functionA(argv):
if argv[0].strip() == "AAA":
return "BBB"
else:
return ""
def functionB(argv):
if argv[0].strip() == "BBB":
try:
return "CCC"+argv[1].strip()
except:
return "CCC"
else:
return ""
def functionC(argv):
if argv[1].strip() == "CCC":
return "DDD"+argv[0].strip()
else:
return ""
patternProcessor = venc2.pattern.processor(".:",":.","::")
patternProcessor.SetFunction("FunctionA", functionA)
patternProcessor.SetFunction("FunctionB", functionB)
patternProcessor.SetFunction("FunctionC", functionC)
patternProcessor.preProcess("test1", ".:FunctionA::AAA:.")
patternProcessor.preProcess("test2", ".:FunctionA::AAB:.")
patternProcessor.preProcess("test3", ".:FunctionB:: .:FunctionA::AAA:. :: DDD :.")
patternProcessor.preProcess("test4", ".:FunctionC:: EEE :: .:FunctionB::BBB:. :.")
patternProcessor.preProcess("test5", ".:FunctionC:: .:FunctionB::BBB:. :: .:FunctionB::BBB:. :.")
patternProcessor.preProcess("test6", ".:FunctionC:: .:FunctionB::BBB:. :: .:FunctionZ::BBB:. :.")
result = ("BBB" == patternProcessor.parse("test1"))
print(result,"\tSimple pattern recognition")
result = ("" == patternProcessor.parse("test2"))
print(result,"\tSimple pattern recognition with wrong parameters")
result = ("CCCDDD" == patternProcessor.parse("test3"))
print(result,"\tRecursive pattern recognition")
result = ("DDDEEE" == patternProcessor.parse("test4"))
print(result,"\tRecursive pattern recognition on second parameter")
result = ("DDDCCC" == patternProcessor.parse("test5"))
print(result,"\tRecursive pattern recognition on first and second parameter")
result = ("" == patternProcessor.parse("test6"))
print(result,"\tRecursive pattern recognition on first and second parameter, with wrong second pattern")
......@@ -42,6 +42,7 @@ from venc2.patterns.processor import PreProcessor
from venc2.threads.categories import CategoriesThread
from venc2.threads.dates import DatesThread
from venc2.threads.entries import EntriesThread
from venc2.threads.feed import FeedThread
from venc2.threads.main import MainThread
from venc2.commands.remote import remote_copy
......@@ -175,14 +176,20 @@ def export_blog(argv=list()):
entry.html_wrapper.below = PreProcessor(''.join(processor.batch_process(entry.html_wrapper.below, "entry.html", False).sub_strings))
entry.rss_wrapper = deepcopy(theme.rss_entry)
entry.rss_wrapper.above = PreProcessor(''.join(processor.batch_process(entry.rss_wrapper.above, "rssEntry.html", False).sub_strings))
entry.rss_wrapper.below = PreProcessor(''.join(processor.batch_process(entry.rss_wrapper.below, "rssEntry.html", False).sub_strings))
entry.rss_wrapper.above = PreProcessor(''.join(processor.batch_process(entry.rss_wrapper.above, "rssEntry.xml", False).sub_strings))
entry.rss_wrapper.below = PreProcessor(''.join(processor.batch_process(entry.rss_wrapper.below, "rssEntry.xml", False).sub_strings))
entry.atom_wrapper = deepcopy(theme.atom_entry)
entry.atom_wrapper.above = PreProcessor(''.join(processor.batch_process(entry.atom_wrapper.above, "atomEntry.xml", False).sub_strings))
entry.atom_wrapper.below = PreProcessor(''.join(processor.batch_process(entry.atom_wrapper.below, "atomEntry.xml", False).sub_strings))
processor.forbidden = non_contextual_pattern_names_entry_keys
theme.header = PreProcessor(''.join(processor.batch_process(theme.header, "header.html").sub_strings))
theme.footer = PreProcessor(''.join(processor.batch_process(theme.footer, "footer.html").sub_strings))
theme.rssHeader = PreProcessor(''.join(processor.batch_process(theme.rss_header, "rssHeader.html").sub_strings))
theme.rssFooter = PreProcessor(''.join(processor.batch_process(theme.rss_footer, "rssFooter.html").sub_strings))
theme.rss_header = PreProcessor(''.join(processor.batch_process(theme.rss_header, "rssHeader.xml").sub_strings))
theme.rss_footer = PreProcessor(''.join(processor.batch_process(theme.rss_footer, "rssFooter.xml").sub_strings))
theme.atom_header = PreProcessor(''.join(processor.batch_process(theme.atom_header, "atomHeader.xml").sub_strings))
theme.atom_footer = PreProcessor(''.join(processor.batch_process(theme.atom_footer, "atomFooter.xml").sub_strings))
# cleaning directory
shutil.rmtree("blog", ignore_errors=False, onerror=rm_tree_error_handler)
......@@ -191,7 +198,6 @@ def export_blog(argv=list()):
datastore.embed_providers = {}
# Starting second pass and exporting
thread = MainThread(messages.export_main_thread, datastore, theme, contextual_pattern_names, non_contextual_pattern_names_entry_keys)
thread.do()
......@@ -205,7 +211,10 @@ def export_blog(argv=list()):
if not datastore.blog_configuration["disable_single_entries"]:
thread = EntriesThread(messages.export_single_entries, datastore, theme, contextual_pattern_names, non_contextual_pattern_names_entry_keys)
thread.do()
thread.do()
if not datastore.blog_configuration["disable_rss_feed"]:
thread = FeedThread("RSS", datastore, theme, contextual_pattern_names, non_contextual_pattern_names_entry_keys, "rss")
thread.do()
# Copy assets and extra files
......
......@@ -132,7 +132,8 @@ def new_blog(argv):
"category_directory_name": "{category}",
"dates_directory_name": "%Y-%m",
"entry_file_name": "entry{entry_id}.html",
"rss_file_name": "feed.xml"},
"rss_file_name": "rss.xml",
"atom_file_name": "atom.xml"},
"entries_per_pages": 10,
"columns": 1,
"feed_lenght": 5,
......
......@@ -187,6 +187,9 @@ class DataStore:
return self.entries[self.requested_entry_index].date.strftime(self.blog_configuration["path"]["dates_directory_name"])
def get_entry_url(self, argv=list()):
if self.blog_configuration["disable_single_entries"]:
return ''
return self.entries[self.requested_entry_index].url
def get_author_name(self, argv=list()):
......@@ -219,8 +222,12 @@ class DataStore:
def for_blog_dates(self, argv):
key = ''.join(argv)
if not key in self.html_blog_dates.keys():
dates = [o for o in self.blog_dates if o["value"] not in self.disable_threads]
self.html_blog_dates[key] = merge(dates, argv)
if self.blog_configuration["disable_archives"]:
self.html_blog_dates[key] = ''
else:
dates = [o for o in self.blog_dates if o["value"] not in self.disable_threads]
self.html_blog_dates[key] = merge(dates, argv)
return self.html_blog_dates[key]
......@@ -260,13 +267,17 @@ class DataStore:
def tree_for_entry_categories(self, argv):
key = ''.join(argv)
if not key in self.entries[self.requested_entry_index].html_categories_tree.keys():
self.entries[self.requested_entry_index].html_categories_tree[key] = self.build_html_categories_tree(
argv[0], #opening_node
argv[1], #opening_branch
argv[2], #closing_branch
argv[3], #closing_node
self.entries[self.requested_entry_index].categories_tree
)
if self.blog_configuration["disable_categories"]:
self.entries[self.requested_entry_index].html_categories_tree[key] = ''
else:
self.entries[self.requested_entry_index].html_categories_tree[key] = self.build_html_categories_tree(
argv[0], #opening_node
argv[1], #opening_branch
argv[2], #closing_branch
argv[3], #closing_node
self.entries[self.requested_entry_index].categories_tree
)
return self.entries[self.requested_entry_index].html_categories_tree[key]
......@@ -274,13 +285,17 @@ class DataStore:
key = ''.join(argv)
# compute once categories tree and deliver baked html
if not key in self.html_categories_tree.keys():
self.html_categories_tree[key] = self.build_html_categories_tree(
argv[0], #opening_node
argv[1], #opening_branch
argv[2], #closing_branch
argv[3], #closing_node
self.entries_per_categories
)
if self.blog_configuration["disable_categories"]:
self.html_categories_tree[key] = ''
else:
self.html_categories_tree[key] = self.build_html_categories_tree(
argv[0], #opening_node
argv[1], #opening_branch
argv[2], #closing_branch
argv[3], #closing_node
self.entries_per_categories
)
return self.html_categories_tree[key]
......@@ -302,23 +317,31 @@ class DataStore:
def leaves_for_entry_categories(self, argv):
key = ''.join(argv)
if not key in self.entries[self.requested_entry_index].html_categories_leaves.keys():
self.entries[self.requested_entry_index].html_categories_leaves[key] = merge(self.entries[self.requested_entry_index].categories_leaves, argv)
if self.blog_configuration["disable_categories"]:
self.entries[self.requested_entry_index].html_categories_leaves[key] = ''
else:
self.entries[self.requested_entry_index].html_categories_leaves[key] = merge(self.entries[self.requested_entry_index].categories_leaves, argv)
return self.entries[self.requested_entry_index].html_categories_leaves[key]
def leaves_for_blog_categories(self, argv):
key = ''.join(argv)
if not key in self.html_categories_leaves.keys():
items = []
for node in self.categories_leaves:
items.append({
"value" : node.value,
"count" : node.count,
"weight" : round(node.weight / self.max_category_weight,2),
"path" : node.path
})
if self.blog_configuration["disable_categories"]:
self.html_categories_leaves[key] = ''
else:
items = []
for node in self.categories_leaves:
items.append({
"value" : node.value,
"count" : node.count,
"weight" : round(node.weight / self.max_category_weight,2),
"path" : node.path
})
self.html_categories_leaves[key] = merge(items, argv)
self.html_categories_leaves[key] = merge(items, argv)
return self.html_categories_leaves[key]
......
......@@ -70,6 +70,7 @@ def get_blog_configuration():
"dates_directory_name",
"entry_file_name",
"rss_file_name",
"atom_file_name",
"ftp",
"entries_sub_folders",
"categories_sub_folders",
......
......@@ -136,10 +136,11 @@ class Entry:
"entry_id": self.id,
"entry_title": self.title
}
sf = paths["entries_sub_folders"].format(**params)
sf = paths["entries_sub_folders"].format(**params).replace(' ','-')
sf = sf+'/' if sf != '' else ''
try:
self.url = ".:GetRelativeOrigin:."+urllib.parse.quote(sf+paths["entry_file_name"].format(**params), encoding=encoding)
except UnicodeEncodeError as e:
self.url = ".:GetRelativeOrigin:."+sf+paths["entry_file_name"].format(**params)
notify("\"{0}\": ".format(sf+paths["entry_file_name"].format(**params))+str(e), color="YELLOW")
......
......@@ -44,7 +44,7 @@ def build_categories_tree(entry_index, input_list, output_tree, output_leaves, m
if node_name == '':
continue
path += node_name+'/'
path += str(node_name+'/').replace(' ','-')
if not node_name in [metadata.value for metadata in root]:
root.append(MetadataNode(node_name, entry_index))
if output_leaves != None and node_name == leave:
......
......@@ -36,11 +36,14 @@ class Theme:
try:
self.header = PreProcessor(open(theme_folder+"chunks/header.html",'r').read())
self.footer = PreProcessor(open(theme_folder+"chunks/footer.html",'r').read())
self.rss_header = PreProcessor(open(theme_folder+"chunks/rssHeader.html",'r').read())
self.rss_footer = PreProcessor(open(theme_folder+"chunks/rssFooter.html",'r').read())
self.rss_header = PreProcessor(open(theme_folder+"chunks/rssHeader.xml",'r').read())
self.rss_footer = PreProcessor(open(theme_folder+"chunks/rssFooter.xml",'r').read())
self.atom_header = PreProcessor(open(theme_folder+"chunks/atomHeader.xml",'r').read())
self.atom_footer = PreProcessor(open(theme_folder+"chunks/atomFooter.xml",'r').read())
self.entry = EntryWrapper(open(theme_folder+"chunks/entry.html",'r').read(), "entry.html")
self.rss_entry = EntryWrapper(open(theme_folder+"chunks/rssEntry.html",'r').read(),"rssEntry.html")
self.rss_entry = EntryWrapper(open(theme_folder+"chunks/rssEntry.xml",'r').read(),"rssEntry.xxml")
self.atom_entry = EntryWrapper(open(theme_folder+"chunks/atomEntry.xml",'r').read(),"atomEntry.xml")
self.audio = open(theme_folder+"chunks/audio.html",'r').read()
self.video = open(theme_folder+"chunks/video.html",'r').read()
......
......@@ -37,6 +37,15 @@ class Thread:
# Setup useful data
self.theme = theme
self.footer = self.theme.footer
self.header = self.theme.header
self.entry = self.theme.entry
self.context_header = "header.html"
self.context_footer = "footer.html"
self.content_type = "html"
self.column_opening = '<div id="__VENC_COLUMN_{0}__" class="__VENC_COLUMN__">'
self.column_closing = "</div>"
self.current_page = 0
self.datastore = datastore
# Setup pattern processor
......@@ -213,7 +222,7 @@ class Thread:
def pre_iteration(self):
self.processor.forbidden = self.forbidden
self.output = ''.join(self.processor.batch_process(self.theme.header, "header.html").sub_strings)
self.output = ''.join(self.processor.batch_process(self.header, self.context_header).sub_strings)
self.processor.forbidden = []
self.columns_number = self.datastore.blog_configuration["columns"]
......@@ -223,10 +232,10 @@ class Thread:
def post_iteration(self):
self.columns_counter = 0
for column in self.columns:
self.output += '<div id="__VENC_COLUMN_'+str(self.columns_counter)+'__" class="__VENC_COLUMN__">'+column+'</div>'
self.output += self.column_opening.format(self.columns_counter)+column+self.column_closing
self.processor.forbidden = self.forbidden
self.output += ''.join(self.processor.batch_process(self.theme.footer, "footer.html").sub_strings)
self.output += ''.join(self.processor.batch_process(self.footer, self.context_footer).sub_strings)
self.write_file(self.output, self.page_number)
......@@ -234,17 +243,17 @@ class Thread:
self.current_page = self.page_number
def do_iteration(self, entry):
self.columns[self.columns_counter] += ''.join(self.processor.batch_process(entry.html_wrapper.above, entry.filename).sub_strings)
self.columns[self.columns_counter] += ''.join(self.processor.batch_process(getattr(entry,self.content_type+"_wrapper").above, entry.filename).sub_strings)
if entry.html_wrapper.required_content_pattern == ".:GetEntryPreview:.":
self.columns[self.columns_counter] += ''.join(self.processor.batch_process(entry.preview, entry.filename,).sub_strings)
self.columns[self.columns_counter] += ''.join(self.processor.batch_process(entry.preview, entry.filename).sub_strings)
elif entry.html_wrapper.required_content_pattern == ".:PreviewIfInThreadElseContent:." and self.in_thread:
self.columns[self.columns_counter] += ''.join(self.processor.batch_process(entry.preview, entry.filename,).sub_strings)
self.columns[self.columns_counter] += ''.join(self.processor.batch_process(entry.preview, entry.filename).sub_strings)
else:
self.columns[self.columns_counter] += ''.join(self.processor.batch_process(entry.content, entry.filename,).sub_strings)
self.columns[self.columns_counter] += ''.join(self.processor.batch_process(entry.content, entry.filename).sub_strings)
self.columns[self.columns_counter] += ''.join(self.processor.batch_process(entry.html_wrapper.below, entry.filename).sub_strings)
self.columns[self.columns_counter] += ''.join(self.processor.batch_process(getattr(entry,self.content_type+"_wrapper").below, entry.filename).sub_strings)
self.columns_counter +=1
if self.columns_counter >= self.columns_number:
......@@ -254,8 +263,8 @@ class Thread:
def do(self):
self.page_number = 0
if self.pages_count == 0:
output = ''.join(self.processor.batch_process(self.theme.header, "header.html").sub_strings)
output += ''.join(self.processor.batch_process(self.theme.footer, "footer.html").sub_strings)
output = ''.join(self.processor.batch_process(self.header, self.context_header).sub_strings)
output += ''.join(self.processor.batch_process(self.footer, self.context_footer).sub_strings)
stream = codecs.open(
self.export_path +'/'+ self.format_filename(0),
'w',
......
......@@ -46,7 +46,7 @@ class CategoriesThread(Thread):
notify("\t"+node.value+"...")
export_path = self.export_path
self.export_path += node.value+'/'
self.export_path += str(node.value+'/').replace(' ','-')
self.relative_origin = ''.join([ '../' for f in self.export_path.split("/")[1:] if f != '' ]).replace("//",'/')
# Get entries
......
......@@ -18,7 +18,6 @@
# along with VenC. If not, see <http://www.gnu.org/licenses/>.
import os
import urllib.parse
from venc2.helpers import notify
from venc2.threads import Thread
......@@ -41,7 +40,7 @@ class DatesThread(Thread):
continue
notify("\t"+thread.value+"...")
self.export_path = "blog/"+self.sub_folders+'/'+thread.value+'/'
self.export_path = str("blog/"+self.sub_folders+'/'+thread.value+'/').replace(' ','-')
os.makedirs(self.export_path)
self.organize_entries([
entry for entry in self.datastore.get_entries_for_given_date(
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment