Nearly There


For most of my life, I have wanted to do one thing. To build a system that lets me build systems for managing content. So many things that I have wanted to build, but I thought about the time needed and would decide against it.

I think within a couple weeks I will have a very amazing system for managing everything that I want to keep track of. It's going to be infinitely expandable, too.

Keep reading to find out more about how it's going to function.

What am I trying to keep track of?

Well, everything.

I know that doesn't say much, but really, that's what I want. 

Like cheese. I love cheese. But I don't often remember exactly which cheeses I like the most. Some are great, some are OK, etc. I also want to make sure that I'm not picking up cheeses that are too similar. So I want just a simple interface where I can keep track of:

  • Name of the cheese
  • Dairy
  • Type
  • Price Range
  • Description
  • Tags
  • Flavors
  • Pairings

Actually, there might be more stuff.

That's really not a ton of stuff to keep track of. A couple of days to put together and make an editor and a view and a search interface. But that's a lot of work for something as frivolous as that. I'd have a much better enjoyment of my cheeses from then on, but is that really worth it?

Another option might be television shows that I watch. I often forget what I've seen, what has new episodes. I don't ever really share what I like with anyone. I don't know who the actors, actresses, writers, directors, etc are. I don't have episode lists. Again, it's a matter of finding the time to put it together. And not just to put it together, but to put it together well.

Making Systems to Create Systems

For several years now, the ideas have been kicking around in my head. And how they would be architected. What things to keep track of. What variables have to be shared. Thinking of editing interfaces, databases, display templates, helpers, models, basically anything that you need to make any one of those things.

Each project at work over the last few years has helped, too. And a lot of the outside projects. Really, it's what I've always done, what I'm always good at doing. And that is looking at hundreds of different things and seeing the patterns that bind them all together. What can be stripped away? How can it be refactored to work better? What makes it more customizable?

Now it's finally started coming together into one final "Unified System of Everything" for content management. The work that I've done in the last month has put all of the pieces into place to finally make the system that I want to make. And it's looking like it's going to be one of the best things I've ever done. 



Page Managers

I had the idea for these several months ago. It's been a recurring issue on a lot of projects. I often have to make managers for page types where the client doesn't have enough budget for us to create a custom interface. So I will make a composer page type and some attributes. Then make the customized view for the front end and just put a link back to composer from the front end. That's essentially how the editing interface for Greaserag works. It's really not intuitive, and I feel a bit guilty actually giving it to the end client. They are usually fine with it, in many ways it's better than other solutions, especially if all you've had is an outdated custom system.

What I'm working on will work a bit like this:

  • Set up your Page Types just like you normally do in Composer. This will set the 'base' settings that are inherited on all sites.
  • Add a new "Page Manager Section" through a dashboard interface. Only people and groups with the correct task permission can do this.
  • Add a new "Page Manager" pages to the different sections through the dashboard interface.

The Page Managers each would have a name, page type, editor form path + package handle, and included content types and order. So you could create "Media" page type with title, description, author, etc. Then make a "TV Shows" page manager section. Underneath that, create a page manager, with Media as the page type, then "Dramas" as the name. The creation form in the dashboard will show all of the options for content items that are available on the master page, and you will be able to turn them off and on, and re-order. There would also be two fields to control what editing form is displayed. One is for the path to an element, and the other is an optional package handle. So you could have the same fields but different edit form layouts for each manager. The options for where to publish the different pages would be there as well, as they are on the standard composer page. So each different manager can have different publishing locations.

The page managers themselves are shown a bit over in the playground, but it's been modified a bit since then:

The buttons are a bit smaller, some things are moved around. But it's still pretty basic.

I'm not sure if basic is the way to go or not, though. I kind of want to modify it so that each one of the manager pages has almost all the features of the existing page search in the dashboard. Only perhaps with a few options removed. I thought that maybe instead of doing a pop up menu, there would be buttons on each row to make it work a bit easier for mobile devices.

There might be options so you could say which fields can be searched, or what the filter for base display is on each Manager. That might just be an option for parent pages, or page type, or whatever. That might change a bit as I finish it up. I haven't figured out exactly how to manage that part of it yet. 

I might change it so that there's a derivative of the page list that also filters by what composer section the page was originally published under? Not entirely sure.

Ajax Page Lists

This I actually just got started on yesterday, but a lot of the pieces have been in the works. It seems like a lot of things that I've made recently have been using ajax for filtering results. Most of them have been based on page types and not blocks though. Which works really well from a coding standpoint, but it's not really the best for re-use. Ideally, it should be something done on the block level.

It's been kind of hard to figure out exactly how to port everything properly to a block. But a lot of that was the fact that a lot of people haven't been actually specifying that they wanted ajax form submission on the filter pages. So trying to make sure that it would work on all different platforms kind of limited me in how I thought about the code and how I approached the interfaces. I'm not sure why I resisted using block ajax urls instead of the prettier ones that come from using pages. In a lot of ways, the page URLs are easier to get and use, but they aren't needed for the stuff to work. 

Anyway, yesterday, I sat down to start in on the front end page list view for a news system that I'm working on for a site at Hutman. It was supposed to just be a simple thumbnail list, but when I looked against the mock ups, I realized that i actually needed to filter it by month and topic as well. And probably with both applying. I had been thinking about making an updated version of Hutman News for quite awhile, it's really been starting to show it's age. Part of that was going to be updating to use the dashboard editing interfaces that I've learned how to do, and the Page Managers stuff fixed that issue. But all of the filtering and display stuff is handled by a page type controller. Which was OK, but not really the best.

It kind of surprised me, actually. Got started around 3, and by about 7:30 or 8 I had all the proof of concept stuff done for porting to a block model. A big key was my recently learning about how you can use data-whatever_var_name to tag extra data onto DOM elements. So I tagged some select elements with data that I needed for submitting to the block controller. That simplified my javascript immensely. Instead of having everything as part of the view in a script tag, it was all in a much smaller external file. 

I could also control a lot more aspects of how the different elements are processed. I can specify that the whole block filter mode will work in multiple mode, allowing more than one field at a time. Then I can specify how a particular index is handled, so topic could be single or multiple. Then you can also specify if all of the filters add or reduce the number of results. Not sure exactly what to call that. But it's all right there on the form fields. The javascript automatically combines it to properly filter.

The ajax filtering is stored in a session variable for each block. So all of the different blocks on the site filter independently. 

I'm going to set it up so that there's a filter block and a display block. The display would have a field in the edit form that would allow you to have a unique ID for each. The filer would then have a dropdown that shows all the named display blocks on the page that you're editing. That would determine what list updates according to the filters. The other option would be what edit form to display. Or possibly you could create custom forms by picking what fields to search against? Not entirely sure what the final format for this will look like, but whatever it is, it's going to be really easy to create new filter forms. This is an example of code to create a select menu:

defined('C5_EXECUTE') or die("Access Denied.");
	$form = Loader::helper('form');
	$c = Page::getCurrentPage();
	$nh = Loader::helper('navigation');
	if (intval($_GET['ccm_paging_p']) > 0) {
		$redirect = 1;
		$redirect_url = $nh->getLinkToCollection($c, true);
	} else {
		$redirect = 0;
		$redirect_url = "";
	$ak = CollectionAttributeKey::getByHandle('news_topic');
	$akc = $ak->getController();
	if (method_exists($akc, 'getOptionUsageArray')) {
		if ($cParentID) {
			$pp = Page::getByID($cParentID);
		} else {
			$pp = false;
		$categoryOptions = $akc->getOptionUsageArray($pp);
		foreach ($categoryOptions as $option) {
			$categoriesArray[$option->getSelectAttributeOptionID()] = $option->getSelectAttributeOptionValue();
		$categoriesArray = array_reverse($categoriesArray, true); 
		$categoriesArray["clear"] = t("Search By Topic");
		$categoriesArray = array_reverse($categoriesArray, true); 
	} else {
		$categoriesArray = false;
	<div id="ajax-page-list-<?= $bID; ?>" class="ajax-page-list-filters">
		<div class="span-1">
		<?= $form->select(
				  'data-filter_column' => 'ak_news_topic',
				  'data-comparison' => 'LIKE',
				  'data-block_filter_mode' => 'multiple',
				  'data-column_filter_mode' => 'single',
				  'data-column_match_mode' => 'strict',
				  'data-filter_page_list_url' => $filter_page_list_url,
				  'data-sort_page_list_url' => $sort_page_list_url,
				  'data-bID' => $bID,
				  'class' => 'ajax-page-list-filter-select'
		<div class="clearfix"></div>

I'm not sure about other developers, but to me that feels like a very handy and easy way to mark up inputs for ajax filtering of a page list. Like I said earlier, it's only about 4 hours into development, so it's not very well fleshed out yet. It only does selects, but that's mostly just because I haven't really tested it with anything else yet. The code for the filtering functions should work just as well with check boxes and text inputs. Dates will probably require some work, but probably not a lot. If I do some reverse engineering of the search forms in the dashboard, I might be able to just use the attribute's form view to get the edit views. I think having it be more exactly specified by the person making the filter block is probably the best way to go, though. I like control over how my interfaces look and work.

Oembed Content Replacement

This is something that I've posted a lot on already, so I won't go into it too much. But it is a pretty key ingredient to making the more 'dashboardy' editing experience work. If you have to use a whole bunch of different types of content, the system starts to break down a little bit. But if you can get around that by allowing people to embed a lot of content with only a few block and attribute types, then things work much, much better. You start having a completely different range of content that you can add with very simple editing interfaces.

More Advanced Composer Integration

As I was working on Oembed, I wanted to put together some test blog posts and test replacing blocks on the built in page type defaults with my Oembed blocks. When I got to looking at the view of those blocks in composer, I was pretty excited that my forms appeared to work fine without much modification. The show/hide stuff and the tabs worked fine. 

But then I tried saving it. And then I learned that you have to modify the field names used in your composer.php file in the block directory. Once I learned that, it made me realize that I can now make almost anything work in composer properly. I'm going to be updating the code for Attributes Slider to work in composer like that. So that you can manage files and everything right there, instead of doing it in a pop up that then shows the view every time it saves. That will be a lot more user friendly. 

I think actually it might drive a lot of other developers to update their blocks to be more composer compatible in order to integrate with this system. Which will be better not just people that use my system, but everyone using concrete5. 

Integrated Theme and File Management Scheme

smart_package_structure.pngOne thing that is always kind of weird is trying to keep things consistent across different packages. It's been an issue of a lot of discussion on the IRC, Peer Review Board and Leaders Forums for concrete5. There, it's mostly how to handle shared javascript and css assets. I want to make a "Common Tools" package that will hold all the different helpers and libraries that are shared across the rest of these tools. Figuring out exactly how to work with the limitations of licensing and the marketplace is going a bit slowly.

When I was working on, I came up with the theme structure that I used for the old version of this website. That used a block to let you control what package elements were loaded to make the php / html DOM. It was clunky and pretty slow, but decent for sketching out themes that follow common patterns. But without the need to make 100s of themes that could all be applied to any one of 10,000 sites and then finely customized on a per-site basis, it never really saw much use.

Outside of that, though, I've been trying to think of all my add ons in terms of how they will be overwritten by other packages. How the code will be read and managed by other developers who have to work with it.

Again, it's still in the formative stages, but I'm starting to keep a very defined folder structure for my elements, models, etc. 

As you can see in the image of my structure, there's a lot of repetition in the naming. My package is page_managers, and then I repeat that inside of the repeated system folders inside of my package. 

Why on earth do you do that?

package_with_overrides.pngI know a lot of people will ask that. One reason is for overriding from the outer directory - if all your elements are grouped under the name of your package, the root override section stays understandable even if you are customizing dozens of packages. The same concept helps keep secondary packages cleaner. 

I've started doing a package per-site, as well. So 'client_name' as the package handle, or something similar. If my overrides in that package are kept to the same convention, then things work much much nicer. 

Take a look at the second image. You can see which packages are being overridden by this package, and where the files relevant to this package are located.

What makes elements better than $this->inc()?

Well, they allow you to pass pre-formatted vars between different content and form layouts. And use those layouts pretty much anywhere. It's also not very, um, 'unit specific' if that makes sense. A theme might need to include a header file and whatever, but other items usually work better loaded from the outer elements directory.

So if you customization options for an integrated package like Oembed, you can use the same form in the dashboard, or a content block, or a page list block. Or someone making their own package that integrates with yours could also include it. Most of the time, I try to also make a database model that will save the data from the form against a supplied block ID, and load it back by block ID, too. It seems to work really well.

Another thing that you can do is really, really simplify your views. For a page list, I might include an element to filter, then one to output the page list. Inside of the one that is passed the page list will be a bit of logic as needed, then the page array is looped over. Normally, we'd be doing this in the block view, but now we're in the package element so you don't have to see it in the block. You can also easily see which package and file is being displayed. 

But you can take it one level further than that. Say you have a block options object. Pass that and the page into another package element. That handles the specific output for that page. Another step, and you could have a switch case to include a different template for different page types.

What you need to realize is that you're not just creating paths for your code, but paths for other developers that use your code. 

Organizing like this also makes development and re-use quite a lot easier, too. Want to move something into your new package and modify? It's all there, quick and easy to find. The output is grouped into logical levels of complexity that make it easier to understand how it will fit into the DOM semantically. 

Not everyone is going to be able to be quite this rigid, but as I mentioned, I'm kind of good at thinking in repeating patterns.

The Sum of All the Parts

Putting all of these things together, it feels like I'm right on the cusp of being able to start making some really sophisticated page applications quite soon. The way that everything is set up, it should be very, very easy to create APIs that connect all of them together. So if all of the 'core' pieces are there, then you could literally have a package that is nothing more than custom templates and a controller but enables a complete application like a news system or a forum.

I'm really looking forward to seeing what comes out of the finished product. It seems like it could really explode the possibilities for concrete5. I have ideas for a lot more things that will integrate and expand on the little bit I've shown here. 

Coming Up:

  • Google Map Page List - Use a geolocation attribute to display pages on a jquery.maps google calendar. Expands on the ajax page list, but uses either an attribute, area, or get_page_info_window() function on the page controller to get the details for a popup when clicked on. Adding and editing would be as easy as editing a standard page list block.
  • Custom Image Gallery Generator - Think Attributes Slider meets Designer Content. I want to move the attributes to a handler class attached to the block instance and the file instance, not just the file instance. So they could be different for each block. And I'd like to allow you to modify which ones are shown on each block, and in which order. On top of that, I'd like to make a core that can be expanded on like the core blocks are, so that you can easily make a dashboard page where you pick what javascript files and css files and whatever you need to use, specify a package name, and boom - you have an image gallery shell. Just write your view and you're done. 
  • Advanced Comment Systems - I might end up just waiting for this to come out with the conversations module or whatever in 5.7. But I kind of envision a comments button on the page managers list view that pops up a manager, tons of different templates, etc.
  • Page Sets - Pick multiple pages to string into a paginated or 'view all' format popular on news sites.
  • Advanced Analytics on Ajax Search - One of the things I'm doing with the ajax searches is keeping them in a logical structure in the session so that they can be serialized and stored, shared, and retrieved. This is great for sharing product or location searches, or really anything. I'm not 100% sure how to keep track of it, but I'd like to start doing reporting on what configurations people share and save. That's especially important if you allow people to say, index a search of content. Do they like to see content about x, y, z, and .porn? Who else does? What other metadata does that tie into?
  • Advanced Categories Attribute - I have an idea for kind of an infinite hierarchy where you can add a new option, then pick what parent it goes with. So you could have "Drama" in 15 different places, but they would actually be different. One could be under "Mid Century American" and another under "Anime." I also want to tie that in with a better search interface that's easier to customize and has better indexing URLs.
  • Wordpress Import - I've done this once. It took a lot to figure out all the different parts to actually get that "once" complete. 90% is still failure. So is 99%. If there are problems with the content that is imported and you are importing 1000s of pages of content, you might get a bit annoyed. I want to be the first person to offer a truly seamless transition.
  • Demo Blog Package - This will show how to use all the different components to create a basic blog from a package installer.
  • Tutorial Videos - I'm going to try to document most features.
  • World Domination - You knew this was in here, right?
  • More stuff. I'm sure more stuff. I keep thinking of more stuff.

But it's 2:56am. I should probably go to bed. I have stuff to do tomorrow.


Like one, maybe two things. :)